[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:44:35 dist-test-slave-h3ft krb5kdc[18627](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:44:35 dist-test-slave-h3ft krb5kdc[18627](info): set up 2 sockets
Aug 06 22:44:35 dist-test-slave-h3ft krb5kdc[18627](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260806 22:44:37.160239 18615 mini_kdc.cc:121] Time spent starting KDC: real 1.322s user 0.001s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Aug 06 22:44:37 dist-test-slave-h3ft krb5kdc[18627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:44:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:44:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.45.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab.
Entry for principal kudu/127.18.45.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab.
I20260806 22:44:37.312026 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:37813
--webserver_interface=127.18.45.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:36297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.45.254:37813
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:44:37.413576 18645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:37.413913 18645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:37.413957 18645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:37.415719 18645 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:44:37.415840 18645 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:37.415874 18645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:37.416039 18645 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:44:37.416105 18645 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:44:37.418282 18645 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:36297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.45.254:37813
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:37813
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.45.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18645
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:44:37.418762 18645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:37.419198 18645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:37.426548 18653 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:37.426534 18657 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:37.427191 18652 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:37.427616 18645 server_base.cc:1034] running on GCE node
I20260806 22:44:37.428071 18645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:37.428579 18645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:37.429752 18645 hybrid_clock.cc:648] HybridClock initialized: now 1786056277429688 us; error 88 us; skew 500 ppm
Aug 06 22:44:37 dist-test-slave-h3ft krb5kdc[18627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056277, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:37.433014 18645 init.cc:377] Logged in from keytab as kudu/127.18.45.254@KRBTEST.COM (short username kudu)
I20260806 22:44:37.433770 18645 webserver.cc:460] Webserver started at http://127.18.45.254:39161/ using document root <none> and password file <none>
I20260806 22:44:37.434154 18645 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:37.434238 18645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:37.434494 18645 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:44:37.435989 18645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/minicluster-data/master-0/data/instance:
uuid: "412402ac979949bbb1e8a5aaeaafa5e7"
format_stamp: "Formatted at 2026-08-06 22:44:37 on dist-test-slave-h3ft"
server_key: "802081260bd33acf28701e4ad9a719f7"
server_key_iv: "ae3bd8cde1ad437ef540a5a143c599da"
server_key_version: "encryptionkey@0"
I20260806 22:44:37.436448 18645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/minicluster-data/master-0/wal/instance:
uuid: "412402ac979949bbb1e8a5aaeaafa5e7"
format_stamp: "Formatted at 2026-08-06 22:44:37 on dist-test-slave-h3ft"
server_key: "802081260bd33acf28701e4ad9a719f7"
server_key_iv: "ae3bd8cde1ad437ef540a5a143c599da"
server_key_version: "encryptionkey@0"
I20260806 22:44:37.438565 18645 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.002s
I20260806 22:44:37.439821 18668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:37.440212 18645 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:44:37.440340 18645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/minicluster-data/master-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/minicluster-data/master-0/wal
uuid: "412402ac979949bbb1e8a5aaeaafa5e7"
format_stamp: "Formatted at 2026-08-06 22:44:37 on dist-test-slave-h3ft"
server_key: "802081260bd33acf28701e4ad9a719f7"
server_key_iv: "ae3bd8cde1ad437ef540a5a143c599da"
server_key_version: "encryptionkey@0"
I20260806 22:44:37.440419 18645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:37.455266 18645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:37.457566 18645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:37.457757 18645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:37.464375 18645 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.254:37813
I20260806 22:44:37.464489 18733 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.254:37813 every 8 connection(s)
I20260806 22:44:37.465168 18645 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
I20260806 22:44:37.466979 18735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:37.470189 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 18645
I20260806 22:44:37.470345 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/minicluster-data/master-0/wal/instance
I20260806 22:44:37.470903 18735 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 412402ac979949bbb1e8a5aaeaafa5e7: Bootstrap starting.
I20260806 22:44:37.471119 18615 external_mini_cluster.cc:1428] Setting key aa0aab0c21f910e5025a3460f38d33dd
I20260806 22:44:37.472188 18735 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 412402ac979949bbb1e8a5aaeaafa5e7: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:37.472875 18735 log.cc:824] T 00000000000000000000000000000000 P 412402ac979949bbb1e8a5aaeaafa5e7: Log is configured to *not* fsync() on all Append() calls
I20260806 22:44:37.473901 18735 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 412402ac979949bbb1e8a5aaeaafa5e7: No bootstrap required, opened a new log
I20260806 22:44:37.476168 18735 raft_consensus.cc:348] T 00000000000000000000000000000000 P 412402ac979949bbb1e8a5aaeaafa5e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "412402ac979949bbb1e8a5aaeaafa5e7" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 37813 } }
I20260806 22:44:37.476388 18735 raft_consensus.cc:374] T 00000000000000000000000000000000 P 412402ac979949bbb1e8a5aaeaafa5e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:37.476481 18735 raft_consensus.cc:729] T 00000000000000000000000000000000 P 412402ac979949bbb1e8a5aaeaafa5e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 412402ac979949bbb1e8a5aaeaafa5e7, State: Initialized, Role: FOLLOWER
I20260806 22:44:37.476706 18735 consensus_queue.cc:260] T 00000000000000000000000000000000 P 412402ac979949bbb1e8a5aaeaafa5e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "412402ac979949bbb1e8a5aaeaafa5e7" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 37813 } }
I20260806 22:44:37.476859 18735 raft_consensus.cc:388] T 00000000000000000000000000000000 P 412402ac979949bbb1e8a5aaeaafa5e7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:44:37.476918 18735 raft_consensus.cc:482] T 00000000000000000000000000000000 P 412402ac979949bbb1e8a5aaeaafa5e7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:44:37.476979 18735 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 412402ac979949bbb1e8a5aaeaafa5e7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:37.478202 18735 raft_consensus.cc:504] T 00000000000000000000000000000000 P 412402ac979949bbb1e8a5aaeaafa5e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "412402ac979949bbb1e8a5aaeaafa5e7" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 37813 } }
I20260806 22:44:37.478484 18735 leader_election.cc:302] T 00000000000000000000000000000000 P 412402ac979949bbb1e8a5aaeaafa5e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 412402ac979949bbb1e8a5aaeaafa5e7; no voters:
I20260806 22:44:37.478785 18735 leader_election.cc:288] T 00000000000000000000000000000000 P 412402ac979949bbb1e8a5aaeaafa5e7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:44:37.478824 18741 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 412402ac979949bbb1e8a5aaeaafa5e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:44:37.479089 18741 raft_consensus.cc:686] T 00000000000000000000000000000000 P 412402ac979949bbb1e8a5aaeaafa5e7 [term 1 LEADER]: Becoming Leader. State: Replica: 412402ac979949bbb1e8a5aaeaafa5e7, State: Running, Role: LEADER
I20260806 22:44:37.479274 18741 consensus_queue.cc:237] T 00000000000000000000000000000000 P 412402ac979949bbb1e8a5aaeaafa5e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "412402ac979949bbb1e8a5aaeaafa5e7" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 37813 } }
I20260806 22:44:37.479496 18735 sys_catalog.cc:564] T 00000000000000000000000000000000 P 412402ac979949bbb1e8a5aaeaafa5e7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:44:37.479990 18742 sys_catalog.cc:455] T 00000000000000000000000000000000 P 412402ac979949bbb1e8a5aaeaafa5e7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "412402ac979949bbb1e8a5aaeaafa5e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "412402ac979949bbb1e8a5aaeaafa5e7" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 37813 } } }
I20260806 22:44:37.480077 18742 sys_catalog.cc:458] T 00000000000000000000000000000000 P 412402ac979949bbb1e8a5aaeaafa5e7 [sys.catalog]: This master's current role is: LEADER
I20260806 22:44:37.480258 18743 sys_catalog.cc:455] T 00000000000000000000000000000000 P 412402ac979949bbb1e8a5aaeaafa5e7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 412402ac979949bbb1e8a5aaeaafa5e7. Latest consensus state: current_term: 1 leader_uuid: "412402ac979949bbb1e8a5aaeaafa5e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "412402ac979949bbb1e8a5aaeaafa5e7" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 37813 } } }
I20260806 22:44:37.480345 18743 sys_catalog.cc:458] T 00000000000000000000000000000000 P 412402ac979949bbb1e8a5aaeaafa5e7 [sys.catalog]: This master's current role is: LEADER
I20260806 22:44:37.480676 18746 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:44:37.482465 18746 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:44:37.485967 18746 catalog_manager.cc:1347] Generated new cluster ID: 3b21f812478a427889e9ce10afec44e7
I20260806 22:44:37.486061 18746 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:44:37 dist-test-slave-h3ft krb5kdc[18627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:44:37.493446 18738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:37.472650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39600 (local address 127.18.45.254:37813)
0806 22:44:37.473010 (+ 360us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:37.473023 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:37.473354 (+ 331us) server_negotiation.cc:407] Connection header received
0806 22:44:37.476141 (+ 2787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:37.476148 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:37.476209 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:37.476403 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:37.478560 (+ 2157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:37.480237 (+ 1677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:37.482339 (+ 2102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:37.483157 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:37.483309 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:37.488493 (+ 5184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:37.488498 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:37.488499 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:37.488517 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:37.490811 (+ 2294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:37.491501 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:37.491505 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:37.491506 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:37.491666 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:37.492008 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:37.492014 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:37.492015 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:37.492325 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:37.492608 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:37.492907 (+ 299us) server_negotiation.cc:299] Negotiation successful
0806 22:44:37.493082 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":150,"threads_started":1}
I20260806 22:44:37.497161 18746 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:44:37.498199 18746 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:44:37.503536 18746 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 412402ac979949bbb1e8a5aaeaafa5e7: Generated new TSK 0
I20260806 22:44:37.503773 18746 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.45.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab.
Entry for principal kudu/127.18.45.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab.
I20260806 22:44:37.546236 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.193:0
--local_ip_for_outbound_sockets=127.18.45.193
--webserver_interface=127.18.45.193
--webserver_port=0
--tserver_master_addrs=127.18.45.254:37813
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab
--principal=kudu/127.18.45.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:36297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:44:37.673359 18772 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:37.673650 18772 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:37.673681 18772 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:37.675109 18772 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:37.675199 18772 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:37.675251 18772 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.193
I20260806 22:44:37.676976 18772 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:36297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab
--principal=kudu/127.18.45.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.45.193
--webserver_port=0
--tserver_master_addrs=127.18.45.254:37813
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18772
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.193
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:44:37.677465 18772 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:37.677771 18772 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:37.685293 18778 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:37.685320 18779 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:37.685503 18781 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:37.685786 18772 server_base.cc:1034] running on GCE node
I20260806 22:44:37.686086 18772 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:37.686424 18772 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:37.687626 18772 hybrid_clock.cc:648] HybridClock initialized: now 1786056277687597 us; error 44 us; skew 500 ppm
Aug 06 22:44:37 dist-test-slave-h3ft krb5kdc[18627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056277, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:37.691284 18772 init.cc:377] Logged in from keytab as kudu/127.18.45.193@KRBTEST.COM (short username kudu)
I20260806 22:44:37.691960 18772 webserver.cc:460] Webserver started at http://127.18.45.193:34373/ using document root <none> and password file <none>
I20260806 22:44:37.692245 18772 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:37.692319 18772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:37.692505 18772 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:44:37.693872 18772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/minicluster-data/ts-0/data/instance:
uuid: "48c4d43d6e5442e6abc4c923737ca9a3"
format_stamp: "Formatted at 2026-08-06 22:44:37 on dist-test-slave-h3ft"
server_key: "f868d684f9da25a06103842f26939ed3"
server_key_iv: "0aad624f45b787b0a43b93b5348ac013"
server_key_version: "encryptionkey@0"
I20260806 22:44:37.694361 18772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/minicluster-data/ts-0/wal/instance:
uuid: "48c4d43d6e5442e6abc4c923737ca9a3"
format_stamp: "Formatted at 2026-08-06 22:44:37 on dist-test-slave-h3ft"
server_key: "f868d684f9da25a06103842f26939ed3"
server_key_iv: "0aad624f45b787b0a43b93b5348ac013"
server_key_version: "encryptionkey@0"
I20260806 22:44:37.696272 18772 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.004s
I20260806 22:44:37.697230 18790 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:37.697664 18772 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:44:37.697799 18772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/minicluster-data/ts-0/wal
uuid: "48c4d43d6e5442e6abc4c923737ca9a3"
format_stamp: "Formatted at 2026-08-06 22:44:37 on dist-test-slave-h3ft"
server_key: "f868d684f9da25a06103842f26939ed3"
server_key_iv: "0aad624f45b787b0a43b93b5348ac013"
server_key_version: "encryptionkey@0"
I20260806 22:44:37.697887 18772 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:37.718894 18772 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:37.721947 18772 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:37.722147 18772 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:37.722564 18772 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:44:37.723061 18772 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:44:37.723153 18772 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:37.723186 18772 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:44:37.723201 18772 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:37.733721 18772 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.193:34503
I20260806 22:44:37.733795 18941 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.193:34503 every 8 connection(s)
I20260806 22:44:37.734266 18772 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
I20260806 22:44:37.735396 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 18772
I20260806 22:44:37.735527 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/minicluster-data/ts-0/wal/instance
I20260806 22:44:37.735769 18615 external_mini_cluster.cc:1428] Setting key d242fcaed3f00f8a4b29ae050cb9b4f9
Aug 06 22:44:37 dist-test-slave-h3ft krb5kdc[18627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056277, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.193@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.45.194@KRBTEST.COM; defaulting to no policy
I20260806 22:44:37.748361 18738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:37.735793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:38921 (local address 127.18.45.254:37813)
0806 22:44:37.735887 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:37.735890 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:37.736244 (+ 354us) server_negotiation.cc:407] Connection header received
0806 22:44:37.737065 (+ 821us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:37.737066 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:37.737087 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:37.737134 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:37.738534 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:37.739439 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:37.740785 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:37.741161 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:37.741243 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:37.744566 (+ 3323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:37.744568 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:37.744569 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:37.744580 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:37.746525 (+ 1945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:37.747156 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:37.747158 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:37.747159 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:37.747233 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:37.747529 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:37.747531 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:37.747531 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:37.747713 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:37.747790 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:37.748123 (+ 333us) server_negotiation.cc:299] Negotiation successful
0806 22:44:37.748243 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 22:44:37.748634 18947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:37.735776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:37813 (local address 127.18.45.193:38921)
0806 22:44:37.736074 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:37.736138 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:37.736943 (+ 805us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:37.737221 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:37.737226 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:37.738090 (+ 864us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:37.738395 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:37.738401 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:37.739570 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:37.739571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:37.740681 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:37.740684 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:37.741276 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:37.741277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:37.741452 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:37.742259 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:44:37.742270 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:37.744427 (+ 2157us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:37.746648 (+ 2221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:37.746659 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:37.746662 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:37.747062 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:37.747318 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:37.747319 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:37.747320 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:37.747456 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:37.747786 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:37.747795 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:37.747973 (+ 178us) client_negotiation.cc:769] Sending connection context
0806 22:44:37.748102 (+ 129us) client_negotiation.cc:240] Negotiation successful
0806 22:44:37.748308 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":149,"threads_started":1}
I20260806 22:44:37.749254 18944 heartbeater.cc:347] Connected to a master server at 127.18.45.254:37813
I20260806 22:44:37.749398 18944 heartbeater.cc:464] Registering TS with master...
I20260806 22:44:37.749661 18944 heartbeater.cc:511] Master 127.18.45.254:37813 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.194@KRBTEST.COM" created.
I20260806 22:44:37.750813 18687 ts_manager.cc:194] Registered new tserver with Master: 48c4d43d6e5442e6abc4c923737ca9a3 (127.18.45.193:34503)
I20260806 22:44:37.751678 18687 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.45.193@KRBTEST.COM'} at 127.18.45.193:38921
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab.
Entry for principal kudu/127.18.45.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab.
I20260806 22:44:37.770334 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.194:0
--local_ip_for_outbound_sockets=127.18.45.194
--webserver_interface=127.18.45.194
--webserver_port=0
--tserver_master_addrs=127.18.45.254:37813
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab
--principal=kudu/127.18.45.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:36297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:44:37.887208 18950 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:37.887770 18950 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:37.887873 18950 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:37.889496 18950 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:37.889585 18950 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:37.889642 18950 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.194
I20260806 22:44:37.892119 18950 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:36297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab
--principal=kudu/127.18.45.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.45.194
--webserver_port=0
--tserver_master_addrs=127.18.45.254:37813
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18950
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.194
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:44:37.892700 18950 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:37.893075 18950 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:37.899147 18962 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:37.899156 18958 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:37.899246 18964 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:37.899648 18950 server_base.cc:1034] running on GCE node
I20260806 22:44:37.899860 18950 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:37.900107 18950 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:37.901257 18950 hybrid_clock.cc:648] HybridClock initialized: now 1786056277901241 us; error 33 us; skew 500 ppm
Aug 06 22:44:37 dist-test-slave-h3ft krb5kdc[18627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056277, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:37.904398 18950 init.cc:377] Logged in from keytab as kudu/127.18.45.194@KRBTEST.COM (short username kudu)
I20260806 22:44:37.904992 18950 webserver.cc:460] Webserver started at http://127.18.45.194:40889/ using document root <none> and password file <none>
I20260806 22:44:37.905267 18950 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:37.905342 18950 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:37.905517 18950 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:44:37.906908 18950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/minicluster-data/ts-1/data/instance:
uuid: "605653eee14d427cb037bdf712bdc00e"
format_stamp: "Formatted at 2026-08-06 22:44:37 on dist-test-slave-h3ft"
server_key: "d7a4fda8245fa13866df3413e5464927"
server_key_iv: "925fdd33c23590ea3c2966f6b1065924"
server_key_version: "encryptionkey@0"
I20260806 22:44:37.907449 18950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/minicluster-data/ts-1/wal/instance:
uuid: "605653eee14d427cb037bdf712bdc00e"
format_stamp: "Formatted at 2026-08-06 22:44:37 on dist-test-slave-h3ft"
server_key: "d7a4fda8245fa13866df3413e5464927"
server_key_iv: "925fdd33c23590ea3c2966f6b1065924"
server_key_version: "encryptionkey@0"
I20260806 22:44:37.909412 18950 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 22:44:37.910383 18974 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:37.910594 18950 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:44:37.910718 18950 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/minicluster-data/ts-1/wal
uuid: "605653eee14d427cb037bdf712bdc00e"
format_stamp: "Formatted at 2026-08-06 22:44:37 on dist-test-slave-h3ft"
server_key: "d7a4fda8245fa13866df3413e5464927"
server_key_iv: "925fdd33c23590ea3c2966f6b1065924"
server_key_version: "encryptionkey@0"
I20260806 22:44:37.910808 18950 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:37.930289 18950 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:37.933198 18950 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:37.933411 18950 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:37.933780 18950 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:44:37.934217 18950 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:44:37.934280 18950 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:37.934331 18950 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:44:37.934362 18950 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:37.943322 18950 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.194:42633
I20260806 22:44:37.943394 19138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.194:42633 every 8 connection(s)
I20260806 22:44:37.943795 18950 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
I20260806 22:44:37.949229 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 18950
I20260806 22:44:37.949359 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/minicluster-data/ts-1/wal/instance
I20260806 22:44:37.949596 18615 external_mini_cluster.cc:1428] Setting key fd8ed7820e758b124cf51e39cf6c630d
Aug 06 22:44:37 dist-test-slave-h3ft krb5kdc[18627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056277, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.194@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:44:37.955590 18738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:37.944708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:36097 (local address 127.18.45.254:37813)
0806 22:44:37.944768 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:37.944776 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:37.945142 (+ 366us) server_negotiation.cc:407] Connection header received
0806 22:44:37.945777 (+ 635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:37.945778 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:37.945791 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:37.945850 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:37.946967 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:37.947623 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:37.948734 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:37.948930 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:37.949007 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:37.952001 (+ 2994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:37.952003 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:37.952004 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:37.952014 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:37.953935 (+ 1921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:37.954401 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:37.954403 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:37.954404 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:37.954486 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:37.954760 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:37.954761 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:37.954761 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:37.954905 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:37.954991 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:37.955364 (+ 373us) server_negotiation.cc:299] Negotiation successful
0806 22:44:37.955488 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:44:37.955817 19141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:37.944725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:37813 (local address 127.18.45.194:36097)
0806 22:44:37.944948 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:37.945024 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:37.945674 (+ 650us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:37.945919 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:37.945923 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:37.946619 (+ 696us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:37.946869 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:37.946874 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:37.947720 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:37.947721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:37.948656 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:37.948658 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:37.949054 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:37.949056 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:37.949225 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:37.949984 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:44:37.949994 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:37.951890 (+ 1896us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:37.954033 (+ 2143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:37.954040 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:37.954043 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:37.954303 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:37.954564 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:37.954565 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:37.954565 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:37.954685 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:37.954980 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:37.954988 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:37.955200 (+ 212us) client_negotiation.cc:769] Sending connection context
0806 22:44:37.955339 (+ 139us) client_negotiation.cc:240] Negotiation successful
0806 22:44:37.955536 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":142,"threads_started":1}
I20260806 22:44:37.956624 19139 heartbeater.cc:347] Connected to a master server at 127.18.45.254:37813
I20260806 22:44:37.956764 19139 heartbeater.cc:464] Registering TS with master...
I20260806 22:44:37.957028 19139 heartbeater.cc:511] Master 127.18.45.254:37813 requested a full tablet report, sending...
I20260806 22:44:37.957520 18687 ts_manager.cc:194] Registered new tserver with Master: 605653eee14d427cb037bdf712bdc00e (127.18.45.194:42633)
I20260806 22:44:37.958003 18687 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.45.194@KRBTEST.COM'} at 127.18.45.194:36097
WARNING: no policy specified for kudu/127.18.45.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab.
Entry for principal kudu/127.18.45.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab.
I20260806 22:44:37.983009 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.195:0
--local_ip_for_outbound_sockets=127.18.45.195
--webserver_interface=127.18.45.195
--webserver_port=0
--tserver_master_addrs=127.18.45.254:37813
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab
--principal=kudu/127.18.45.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:36297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:44:38.086807 19145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:38.087177 19145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:38.087232 19145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:38.088601 19145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:38.088675 19145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:38.088730 19145 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.195
I20260806 22:44:38.090416 19145 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:36297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab
--principal=kudu/127.18.45.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.45.195
--webserver_port=0
--tserver_master_addrs=127.18.45.254:37813
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19145
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.195
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:44:38.090741 19145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:38.091110 19145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:38.097749 19158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:38.097874 19151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:38.097757 19155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:38.097749 19145 server_base.cc:1034] running on GCE node
I20260806 22:44:38.098322 19145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:38.098687 19145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:38.099898 19145 hybrid_clock.cc:648] HybridClock initialized: now 1786056278099882 us; error 41 us; skew 500 ppm
Aug 06 22:44:38 dist-test-slave-h3ft krb5kdc[18627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056278, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:38.103626 19145 init.cc:377] Logged in from keytab as kudu/127.18.45.195@KRBTEST.COM (short username kudu)
I20260806 22:44:38.104270 19145 webserver.cc:460] Webserver started at http://127.18.45.195:35679/ using document root <none> and password file <none>
I20260806 22:44:38.104576 19145 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:38.104657 19145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:38.104835 19145 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:44:38.106323 19145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/minicluster-data/ts-2/data/instance:
uuid: "50b8fb300a4146729250a3cef1ac6d8f"
format_stamp: "Formatted at 2026-08-06 22:44:38 on dist-test-slave-h3ft"
server_key: "ba5eb4f55a144245befeac5443c0d0ac"
server_key_iv: "47a28cf37e237e7735953d058a227e8a"
server_key_version: "encryptionkey@0"
I20260806 22:44:38.106837 19145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/minicluster-data/ts-2/wal/instance:
uuid: "50b8fb300a4146729250a3cef1ac6d8f"
format_stamp: "Formatted at 2026-08-06 22:44:38 on dist-test-slave-h3ft"
server_key: "ba5eb4f55a144245befeac5443c0d0ac"
server_key_iv: "47a28cf37e237e7735953d058a227e8a"
server_key_version: "encryptionkey@0"
I20260806 22:44:38.108603 19145 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:44:38.109624 19165 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:38.109879 19145 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:44:38.110055 19145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/minicluster-data/ts-2/wal
uuid: "50b8fb300a4146729250a3cef1ac6d8f"
format_stamp: "Formatted at 2026-08-06 22:44:38 on dist-test-slave-h3ft"
server_key: "ba5eb4f55a144245befeac5443c0d0ac"
server_key_iv: "47a28cf37e237e7735953d058a227e8a"
server_key_version: "encryptionkey@0"
I20260806 22:44:38.110167 19145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:38.154409 19145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:38.157226 19145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:38.157449 19145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:38.157795 19145 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:44:38.158226 19145 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:44:38.158291 19145 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:38.158340 19145 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:44:38.158373 19145 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:38.168526 19145 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.195:44503
I20260806 22:44:38.168622 19323 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.195:44503 every 8 connection(s)
I20260806 22:44:38.169252 19145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
I20260806 22:44:38.171617 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 19145
I20260806 22:44:38.171751 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786056275803658-18615-0/minicluster-data/ts-2/wal/instance
I20260806 22:44:38.171973 18615 external_mini_cluster.cc:1428] Setting key 90749edf703e686f94d4867e69eafa86
Aug 06 22:44:38 dist-test-slave-h3ft krb5kdc[18627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056278, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.195@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:44:38.182796 18738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:38.170446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.195:33477 (local address 127.18.45.254:37813)
0806 22:44:38.170550 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:38.170553 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:38.170853 (+ 300us) server_negotiation.cc:407] Connection header received
0806 22:44:38.171972 (+ 1119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:38.171974 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:38.171993 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:38.172069 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:38.173766 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:38.174634 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:38.176346 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:38.176568 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:38.176636 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:38.179410 (+ 2774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:38.179412 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:38.179412 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:38.179420 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:38.181311 (+ 1891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:38.181788 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:38.181790 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:38.181790 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:38.181855 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:38.182125 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:38.182126 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:38.182126 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:38.182256 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:38.182309 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:38.182634 (+ 325us) server_negotiation.cc:299] Negotiation successful
0806 22:44:38.182714 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 22:44:38.183066 19328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:38.170424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:37813 (local address 127.18.45.195:33477)
0806 22:44:38.170682 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:38.170747 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:38.171758 (+ 1011us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:38.172200 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:38.172205 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:38.173275 (+ 1070us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:38.173652 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:38.173660 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:38.174739 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:38.174740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:38.176271 (+ 1531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:38.176274 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:38.176654 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:38.176655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:38.176787 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:38.177483 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:44:38.177494 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:38.179324 (+ 1830us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:38.181408 (+ 2084us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:38.181416 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:38.181418 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:38.181684 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:38.181933 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:38.181934 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:38.181934 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:38.182055 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:38.182334 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:38.182340 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:38.182496 (+ 156us) client_negotiation.cc:769] Sending connection context
0806 22:44:38.182593 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 22:44:38.182746 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":121,"threads_started":1}
I20260806 22:44:38.183705 19326 heartbeater.cc:347] Connected to a master server at 127.18.45.254:37813
I20260806 22:44:38.183840 19326 heartbeater.cc:464] Registering TS with master...
I20260806 22:44:38.184108 19326 heartbeater.cc:511] Master 127.18.45.254:37813 requested a full tablet report, sending...
I20260806 22:44:38.184576 18687 ts_manager.cc:194] Registered new tserver with Master: 50b8fb300a4146729250a3cef1ac6d8f (127.18.45.195:44503)
I20260806 22:44:38.185011 18687 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.45.195@KRBTEST.COM'} at 127.18.45.195:33477
I20260806 22:44:38.195751 18615 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 22:44:38 dist-test-slave-h3ft krb5kdc[18627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056278, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 22:44:38 dist-test-slave-h3ft krb5kdc[18627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056278, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.45.193@KRBTEST.COM
I20260806 22:44:38.225941 19338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:38.216579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46604 (local address 127.18.45.193:34503)
0806 22:44:38.216769 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:38.216777 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:38.216812 (+ 35us) server_negotiation.cc:407] Connection header received
0806 22:44:38.216850 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:38.216852 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:38.216872 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:38.217001 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:38.217666 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:38.218637 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:38.219479 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:38.219922 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:38.219997 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:38.222685 (+ 2688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:38.222695 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:38.222696 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:38.222704 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:38.224618 (+ 1914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:38.225019 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:38.225023 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:38.225023 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:38.225104 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:38.225349 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:38.225350 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:38.225350 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:38.225524 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:38.225664 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:38.225764 (+ 100us) server_negotiation.cc:299] Negotiation successful
0806 22:44:38.225840 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":79,"threads_started":1}
W20260806 22:44:38.226538 18831 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46604
Aug 06 22:44:38 dist-test-slave-h3ft krb5kdc[18627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056278, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 22:44:38 dist-test-slave-h3ft krb5kdc[18627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056278, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.45.193@KRBTEST.COM
I20260806 22:44:38.256572 19338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:38.247796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46620 (local address 127.18.45.193:34503)
0806 22:44:38.247859 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:38.247861 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:38.247951 (+ 90us) server_negotiation.cc:407] Connection header received
0806 22:44:38.248039 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:38.248041 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:38.248059 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:38.248112 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:38.248751 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:38.249328 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:38.250139 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:38.250363 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:38.250460 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:38.253317 (+ 2857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:38.253319 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:38.253319 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:38.253337 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:38.255165 (+ 1828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:38.255602 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:38.255604 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:38.255604 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:38.255681 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:38.255924 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:38.255925 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:38.255926 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:38.256059 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:38.256184 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:38.256317 (+ 133us) server_negotiation.cc:299] Negotiation successful
0806 22:44:38.256437 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 22:44:38.257385 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 18772
I20260806 22:44:38.271962 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 18950
I20260806 22:44:38.291868 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 19145
I20260806 22:44:38.304672 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 18645
2026-08-06T22:44:38Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2504 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:44:38 dist-test-slave-h3ft krb5kdc[19350](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:44:38 dist-test-slave-h3ft krb5kdc[19350](info): set up 2 sockets
Aug 06 22:44:38 dist-test-slave-h3ft krb5kdc[19350](info): commencing operation
krb5kdc: starting...
W20260806 22:44:39.564080 18615 mini_kdc.cc:121] Time spent starting KDC: real 1.228s user 0.005s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Aug 06 22:44:39 dist-test-slave-h3ft krb5kdc[19350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056279, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:44:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:44:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.45.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab.
Entry for principal kudu/127.18.45.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab.
I20260806 22:44:39.714485 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:44359
--webserver_interface=127.18.45.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:46659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.45.254:44359
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:44:39.811980 19368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:39.812264 19368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:39.812291 19368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:39.813625 19368 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:44:39.813680 19368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:39.813696 19368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:39.813709 19368 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:44:39.813721 19368 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:44:39.815740 19368 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:46659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.45.254:44359
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:44359
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.45.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19368
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:44:39.816102 19368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:39.816437 19368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:39.822379 19380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:39.822396 19377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:39.822377 19378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:39.822821 19368 server_base.cc:1034] running on GCE node
I20260806 22:44:39.823239 19368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:39.823661 19368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:39.824820 19368 hybrid_clock.cc:648] HybridClock initialized: now 1786056279824803 us; error 43 us; skew 500 ppm
Aug 06 22:44:39 dist-test-slave-h3ft krb5kdc[19350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056279, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:39.827912 19368 init.cc:377] Logged in from keytab as kudu/127.18.45.254@KRBTEST.COM (short username kudu)
I20260806 22:44:39.828524 19368 webserver.cc:460] Webserver started at http://127.18.45.254:43711/ using document root <none> and password file <none>
I20260806 22:44:39.828794 19368 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:39.828845 19368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:39.829041 19368 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:44:39.830374 19368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/minicluster-data/master-0/data/instance:
uuid: "3bfba0bd70254e26b817afecc82f261e"
format_stamp: "Formatted at 2026-08-06 22:44:39 on dist-test-slave-h3ft"
server_key: "64d6da9447b9d1b1762cf50403778504"
server_key_iv: "0a4d34bbab600b93c8f041d1fb8f83e7"
server_key_version: "encryptionkey@0"
I20260806 22:44:39.830808 19368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/minicluster-data/master-0/wal/instance:
uuid: "3bfba0bd70254e26b817afecc82f261e"
format_stamp: "Formatted at 2026-08-06 22:44:39 on dist-test-slave-h3ft"
server_key: "64d6da9447b9d1b1762cf50403778504"
server_key_iv: "0a4d34bbab600b93c8f041d1fb8f83e7"
server_key_version: "encryptionkey@0"
I20260806 22:44:39.832654 19368 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:44:39.833626 19387 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:39.833864 19368 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:44:39.833976 19368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/minicluster-data/master-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/minicluster-data/master-0/wal
uuid: "3bfba0bd70254e26b817afecc82f261e"
format_stamp: "Formatted at 2026-08-06 22:44:39 on dist-test-slave-h3ft"
server_key: "64d6da9447b9d1b1762cf50403778504"
server_key_iv: "0a4d34bbab600b93c8f041d1fb8f83e7"
server_key_version: "encryptionkey@0"
I20260806 22:44:39.834084 19368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:39.862386 19368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:39.865243 19368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:39.865450 19368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:39.870731 19368 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.254:44359
I20260806 22:44:39.870764 19460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.254:44359 every 8 connection(s)
I20260806 22:44:39.871271 19368 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
I20260806 22:44:39.872401 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 19368
I20260806 22:44:39.872529 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/minicluster-data/master-0/wal/instance
I20260806 22:44:39.872514 19461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:39.872757 18615 external_mini_cluster.cc:1428] Setting key 4efcf0be6d93fb9b5c06df2e295daf2e
I20260806 22:44:39.876402 19461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3bfba0bd70254e26b817afecc82f261e: Bootstrap starting.
I20260806 22:44:39.877204 19461 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3bfba0bd70254e26b817afecc82f261e: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:39.877549 19461 log.cc:824] T 00000000000000000000000000000000 P 3bfba0bd70254e26b817afecc82f261e: Log is configured to *not* fsync() on all Append() calls
I20260806 22:44:39.878407 19461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3bfba0bd70254e26b817afecc82f261e: No bootstrap required, opened a new log
Aug 06 22:44:39 dist-test-slave-h3ft krb5kdc[19350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056279, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:44:39.880973 19461 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3bfba0bd70254e26b817afecc82f261e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bfba0bd70254e26b817afecc82f261e" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 44359 } }
I20260806 22:44:39.881203 19461 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3bfba0bd70254e26b817afecc82f261e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:39.881248 19461 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3bfba0bd70254e26b817afecc82f261e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bfba0bd70254e26b817afecc82f261e, State: Initialized, Role: FOLLOWER
I20260806 22:44:39.881415 19461 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3bfba0bd70254e26b817afecc82f261e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bfba0bd70254e26b817afecc82f261e" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 44359 } }
I20260806 22:44:39.881525 19461 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3bfba0bd70254e26b817afecc82f261e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:44:39.881589 19461 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3bfba0bd70254e26b817afecc82f261e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:44:39.881649 19461 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3bfba0bd70254e26b817afecc82f261e [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:39.882325 19461 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3bfba0bd70254e26b817afecc82f261e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bfba0bd70254e26b817afecc82f261e" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 44359 } }
I20260806 22:44:39.882453 19461 leader_election.cc:302] T 00000000000000000000000000000000 P 3bfba0bd70254e26b817afecc82f261e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3bfba0bd70254e26b817afecc82f261e; no voters:
I20260806 22:44:39.882699 19461 leader_election.cc:288] T 00000000000000000000000000000000 P 3bfba0bd70254e26b817afecc82f261e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:44:39.882736 19468 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3bfba0bd70254e26b817afecc82f261e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:44:39.882880 19468 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3bfba0bd70254e26b817afecc82f261e [term 1 LEADER]: Becoming Leader. State: Replica: 3bfba0bd70254e26b817afecc82f261e, State: Running, Role: LEADER
I20260806 22:44:39.883068 19461 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3bfba0bd70254e26b817afecc82f261e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:44:39.883060 19468 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3bfba0bd70254e26b817afecc82f261e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bfba0bd70254e26b817afecc82f261e" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 44359 } }
I20260806 22:44:39.883533 19470 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3bfba0bd70254e26b817afecc82f261e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3bfba0bd70254e26b817afecc82f261e. Latest consensus state: current_term: 1 leader_uuid: "3bfba0bd70254e26b817afecc82f261e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bfba0bd70254e26b817afecc82f261e" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 44359 } } }
I20260806 22:44:39.883559 19469 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3bfba0bd70254e26b817afecc82f261e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3bfba0bd70254e26b817afecc82f261e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bfba0bd70254e26b817afecc82f261e" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 44359 } } }
I20260806 22:44:39.883622 19470 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3bfba0bd70254e26b817afecc82f261e [sys.catalog]: This master's current role is: LEADER
I20260806 22:44:39.883658 19469 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3bfba0bd70254e26b817afecc82f261e [sys.catalog]: This master's current role is: LEADER
I20260806 22:44:39.884140 19472 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:44:39.884806 19472 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:44:39.885749 19464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:39.873465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55724 (local address 127.18.45.254:44359)
0806 22:44:39.873701 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:39.873711 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:39.873748 (+ 37us) server_negotiation.cc:407] Connection header received
0806 22:44:39.874354 (+ 606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:39.874363 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:39.874421 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:39.874563 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:39.875517 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:39.876564 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:39.877387 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:39.877884 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:39.878017 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:39.881035 (+ 3018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:39.881040 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:39.881040 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:39.881056 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:39.883336 (+ 2280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:39.883814 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:39.883818 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:39.883819 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:39.883940 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:39.884216 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:39.884217 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:39.884218 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:39.884583 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:39.884744 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:39.885392 (+ 648us) server_negotiation.cc:299] Negotiation successful
0806 22:44:39.885554 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":119,"threads_started":1}
I20260806 22:44:39.886971 19472 catalog_manager.cc:1347] Generated new cluster ID: c709608720884d6fbd00e09b0965f7b9
I20260806 22:44:39.887054 19472 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:44:39.897424 19472 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:44:39.898087 19472 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:44:39.909559 19472 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3bfba0bd70254e26b817afecc82f261e: Generated new TSK 0
I20260806 22:44:39.909837 19472 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.45.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab.
Entry for principal kudu/127.18.45.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab.
I20260806 22:44:39.946712 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.193:0
--local_ip_for_outbound_sockets=127.18.45.193
--webserver_interface=127.18.45.193
--webserver_port=0
--tserver_master_addrs=127.18.45.254:44359
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab
--principal=kudu/127.18.45.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:46659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:44:40.050311 19491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:40.050597 19491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:40.050632 19491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:40.052116 19491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:40.052179 19491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:40.052212 19491 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.193
I20260806 22:44:40.054147 19491 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:46659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab
--principal=kudu/127.18.45.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.45.193
--webserver_port=0
--tserver_master_addrs=127.18.45.254:44359
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19491
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.193
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:44:40.054415 19491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:40.054672 19491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:40.061347 19503 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:40.061347 19500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:40.061412 19491 server_base.cc:1034] running on GCE node
W20260806 22:44:40.061347 19507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:40.062052 19491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:40.062431 19491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:40.063642 19491 hybrid_clock.cc:648] HybridClock initialized: now 1786056280063599 us; error 69 us; skew 500 ppm
Aug 06 22:44:40 dist-test-slave-h3ft krb5kdc[19350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056280, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:40.067042 19491 init.cc:377] Logged in from keytab as kudu/127.18.45.193@KRBTEST.COM (short username kudu)
I20260806 22:44:40.067706 19491 webserver.cc:460] Webserver started at http://127.18.45.193:37845/ using document root <none> and password file <none>
I20260806 22:44:40.068012 19491 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:40.068063 19491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:40.068177 19491 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:44:40.069700 19491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/minicluster-data/ts-0/data/instance:
uuid: "a079331ebcb141fb86b9fe54245bb2b3"
format_stamp: "Formatted at 2026-08-06 22:44:40 on dist-test-slave-h3ft"
server_key: "52a0880fba65f80e207a95bb4fd7ca5c"
server_key_iv: "c1c7be0f326779f9080897f89b36c094"
server_key_version: "encryptionkey@0"
I20260806 22:44:40.070472 19491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/minicluster-data/ts-0/wal/instance:
uuid: "a079331ebcb141fb86b9fe54245bb2b3"
format_stamp: "Formatted at 2026-08-06 22:44:40 on dist-test-slave-h3ft"
server_key: "52a0880fba65f80e207a95bb4fd7ca5c"
server_key_iv: "c1c7be0f326779f9080897f89b36c094"
server_key_version: "encryptionkey@0"
I20260806 22:44:40.072471 19491 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.004s
I20260806 22:44:40.073709 19515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:40.074048 19491 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:44:40.074203 19491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/minicluster-data/ts-0/wal
uuid: "a079331ebcb141fb86b9fe54245bb2b3"
format_stamp: "Formatted at 2026-08-06 22:44:40 on dist-test-slave-h3ft"
server_key: "52a0880fba65f80e207a95bb4fd7ca5c"
server_key_iv: "c1c7be0f326779f9080897f89b36c094"
server_key_version: "encryptionkey@0"
I20260806 22:44:40.074304 19491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:40.090368 19491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:40.093803 19491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:40.094101 19491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:40.094491 19491 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:44:40.094931 19491 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:44:40.094990 19491 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:40.095063 19491 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:44:40.095096 19491 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:40.103217 19491 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.193:34597
I20260806 22:44:40.103273 19672 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.193:34597 every 8 connection(s)
I20260806 22:44:40.103734 19491 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
I20260806 22:44:40.104835 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 19491
I20260806 22:44:40.104948 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/minicluster-data/ts-0/wal/instance
I20260806 22:44:40.105165 18615 external_mini_cluster.cc:1428] Setting key 788aa225904fd2240a50bf9165fde076
Aug 06 22:44:40 dist-test-slave-h3ft krb5kdc[19350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056280, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.193@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:44:40.116699 19464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:40.104810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:44995 (local address 127.18.45.254:44359)
0806 22:44:40.104885 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:40.104888 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:40.105256 (+ 368us) server_negotiation.cc:407] Connection header received
0806 22:44:40.106682 (+ 1426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:40.106683 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:40.106700 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:40.106761 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:40.107652 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:40.108234 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:40.109372 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:40.109728 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:40.109804 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:40.112850 (+ 3046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:40.112853 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:40.112853 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:40.112864 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:40.114850 (+ 1986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:40.115428 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:40.115430 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:40.115430 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:40.115507 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:40.115796 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:40.115798 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:40.115798 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:40.115977 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:40.116070 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:40.116409 (+ 339us) server_negotiation.cc:299] Negotiation successful
0806 22:44:40.116578 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
WARNING: no policy specified for kudu/127.18.45.194@KRBTEST.COM; defaulting to no policy
I20260806 22:44:40.116919 19676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:40.104765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:44359 (local address 127.18.45.193:44995)
0806 22:44:40.105088 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:40.105143 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:40.106554 (+ 1411us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:40.106870 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:40.106875 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:40.107274 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:40.107538 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:40.107543 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:40.108354 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:40.108355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:40.109272 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:40.109274 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:40.109884 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:40.109885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:40.110035 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:40.110781 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:44:40.110791 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:40.112728 (+ 1937us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:40.114959 (+ 2231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:40.114968 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:40.114970 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:40.115334 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:40.115587 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:40.115588 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:40.115588 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:40.115710 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:40.116061 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:40.116066 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:40.116219 (+ 153us) client_negotiation.cc:769] Sending connection context
0806 22:44:40.116349 (+ 130us) client_negotiation.cc:240] Negotiation successful
0806 22:44:40.116558 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":113,"threads_started":1}
I20260806 22:44:40.117664 19673 heartbeater.cc:347] Connected to a master server at 127.18.45.254:44359
I20260806 22:44:40.117812 19673 heartbeater.cc:464] Registering TS with master...
I20260806 22:44:40.118134 19673 heartbeater.cc:511] Master 127.18.45.254:44359 requested a full tablet report, sending...
I20260806 22:44:40.118827 19409 ts_manager.cc:194] Registered new tserver with Master: a079331ebcb141fb86b9fe54245bb2b3 (127.18.45.193:34597)
I20260806 22:44:40.119669 19409 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.45.193@KRBTEST.COM'} at 127.18.45.193:44995
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab.
Entry for principal kudu/127.18.45.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab.
I20260806 22:44:40.139729 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.194:0
--local_ip_for_outbound_sockets=127.18.45.194
--webserver_interface=127.18.45.194
--webserver_port=0
--tserver_master_addrs=127.18.45.254:44359
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab
--principal=kudu/127.18.45.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:46659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:44:40.238098 19681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:40.238404 19681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:40.238435 19681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:40.239840 19681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:40.239897 19681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:40.239926 19681 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.194
I20260806 22:44:40.241679 19681 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:46659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab
--principal=kudu/127.18.45.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.45.194
--webserver_port=0
--tserver_master_addrs=127.18.45.254:44359
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19681
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.194
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:44:40.242005 19681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:40.242309 19681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:40.248281 19691 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:40.248466 19688 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:40.248281 19689 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:40.248579 19681 server_base.cc:1034] running on GCE node
I20260806 22:44:40.248869 19681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:40.249178 19681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:40.250396 19681 hybrid_clock.cc:648] HybridClock initialized: now 1786056280250365 us; error 61 us; skew 500 ppm
Aug 06 22:44:40 dist-test-slave-h3ft krb5kdc[19350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056280, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:40.253679 19681 init.cc:377] Logged in from keytab as kudu/127.18.45.194@KRBTEST.COM (short username kudu)
I20260806 22:44:40.254177 19681 webserver.cc:460] Webserver started at http://127.18.45.194:43813/ using document root <none> and password file <none>
I20260806 22:44:40.254408 19681 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:40.254456 19681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:40.254566 19681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:44:40.255852 19681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/minicluster-data/ts-1/data/instance:
uuid: "1f29d7059482448b9df6fa27758283c3"
format_stamp: "Formatted at 2026-08-06 22:44:40 on dist-test-slave-h3ft"
server_key: "df3569a6f7dbfb9c57243c940185b45c"
server_key_iv: "3f1b86f6e1d5ca9334971e844db72bf4"
server_key_version: "encryptionkey@0"
I20260806 22:44:40.256240 19681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/minicluster-data/ts-1/wal/instance:
uuid: "1f29d7059482448b9df6fa27758283c3"
format_stamp: "Formatted at 2026-08-06 22:44:40 on dist-test-slave-h3ft"
server_key: "df3569a6f7dbfb9c57243c940185b45c"
server_key_iv: "3f1b86f6e1d5ca9334971e844db72bf4"
server_key_version: "encryptionkey@0"
I20260806 22:44:40.257863 19681 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:44:40.258716 19700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:40.259020 19681 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:44:40.259133 19681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/minicluster-data/ts-1/wal
uuid: "1f29d7059482448b9df6fa27758283c3"
format_stamp: "Formatted at 2026-08-06 22:44:40 on dist-test-slave-h3ft"
server_key: "df3569a6f7dbfb9c57243c940185b45c"
server_key_iv: "3f1b86f6e1d5ca9334971e844db72bf4"
server_key_version: "encryptionkey@0"
I20260806 22:44:40.259204 19681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:40.271227 19681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:40.274019 19681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:40.274158 19681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:40.274401 19681 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:44:40.274737 19681 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:44:40.274772 19681 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:40.274798 19681 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:44:40.274813 19681 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:40.282554 19681 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.194:34659
I20260806 22:44:40.282693 19865 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.194:34659 every 8 connection(s)
I20260806 22:44:40.283118 19681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
I20260806 22:44:40.287242 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 19681
I20260806 22:44:40.287371 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/minicluster-data/ts-1/wal/instance
I20260806 22:44:40.287616 18615 external_mini_cluster.cc:1428] Setting key f51f438cddf1d1b67d0e16be2baf9e76
Aug 06 22:44:40 dist-test-slave-h3ft krb5kdc[19350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056280, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.194@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:44:40.295557 19464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:40.284107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:40869 (local address 127.18.45.254:44359)
0806 22:44:40.284207 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:40.284210 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:40.284503 (+ 293us) server_negotiation.cc:407] Connection header received
0806 22:44:40.285719 (+ 1216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:40.285721 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:40.285734 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:40.285802 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:40.286509 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:40.287248 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:40.288781 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:40.289015 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:40.289079 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:40.292058 (+ 2979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:40.292060 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:40.292061 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:40.292071 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:40.293834 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:40.294336 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:40.294338 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:40.294338 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:40.294415 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:40.294700 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:40.294701 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:40.294701 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:40.294844 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:40.294929 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:40.295323 (+ 394us) server_negotiation.cc:299] Negotiation successful
0806 22:44:40.295448 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 22:44:40.295805 19869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:40.284079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:44359 (local address 127.18.45.194:40869)
0806 22:44:40.284300 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:40.284348 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:40.285594 (+ 1246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:40.285864 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:40.285868 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:40.286168 (+ 300us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:40.286398 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:40.286411 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:40.287371 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:40.287373 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:40.288685 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:40.288690 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:40.289116 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:40.289117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:40.289281 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:40.289955 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:44:40.289970 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:40.291938 (+ 1968us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:40.293951 (+ 2013us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:40.293962 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:40.293965 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:40.294241 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:40.294500 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:40.294501 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:40.294501 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:40.294626 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:40.294928 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:40.294938 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:40.295176 (+ 238us) client_negotiation.cc:769] Sending connection context
0806 22:44:40.295301 (+ 125us) client_negotiation.cc:240] Negotiation successful
0806 22:44:40.295499 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":129,"threads_started":1}
I20260806 22:44:40.296568 19867 heartbeater.cc:347] Connected to a master server at 127.18.45.254:44359
I20260806 22:44:40.296728 19867 heartbeater.cc:464] Registering TS with master...
I20260806 22:44:40.296999 19867 heartbeater.cc:511] Master 127.18.45.254:44359 requested a full tablet report, sending...
I20260806 22:44:40.297581 19409 ts_manager.cc:194] Registered new tserver with Master: 1f29d7059482448b9df6fa27758283c3 (127.18.45.194:34659)
I20260806 22:44:40.298185 19409 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.45.194@KRBTEST.COM'} at 127.18.45.194:40869
WARNING: no policy specified for kudu/127.18.45.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab.
Entry for principal kudu/127.18.45.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab.
I20260806 22:44:40.322280 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.195:0
--local_ip_for_outbound_sockets=127.18.45.195
--webserver_interface=127.18.45.195
--webserver_port=0
--tserver_master_addrs=127.18.45.254:44359
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab
--principal=kudu/127.18.45.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:46659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:44:40.436717 19873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:40.437115 19873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:40.437170 19873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:40.439087 19873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:40.439157 19873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:40.439219 19873 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.195
I20260806 22:44:40.442502 19873 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:46659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab
--principal=kudu/127.18.45.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.45.195
--webserver_port=0
--tserver_master_addrs=127.18.45.254:44359
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19873
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.195
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:44:40.442884 19873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:40.443295 19873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:40.451332 19884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:40.451547 19873 server_base.cc:1034] running on GCE node
W20260806 22:44:40.451436 19887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:40.451316 19883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:40.451962 19873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:40.452266 19873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:40.453408 19873 hybrid_clock.cc:648] HybridClock initialized: now 1786056280453382 us; error 54 us; skew 500 ppm
Aug 06 22:44:40 dist-test-slave-h3ft krb5kdc[19350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056280, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:40.457253 19873 init.cc:377] Logged in from keytab as kudu/127.18.45.195@KRBTEST.COM (short username kudu)
I20260806 22:44:40.458002 19873 webserver.cc:460] Webserver started at http://127.18.45.195:44097/ using document root <none> and password file <none>
I20260806 22:44:40.458411 19873 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:40.458511 19873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:40.458696 19873 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:44:40.460762 19873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/minicluster-data/ts-2/data/instance:
uuid: "38c55fab9ce14140bcc2c24db6fbba75"
format_stamp: "Formatted at 2026-08-06 22:44:40 on dist-test-slave-h3ft"
server_key: "4cd75429b80cffda151828fb87406e08"
server_key_iv: "7875314836554e622aa04926f5c3601e"
server_key_version: "encryptionkey@0"
I20260806 22:44:40.461377 19873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/minicluster-data/ts-2/wal/instance:
uuid: "38c55fab9ce14140bcc2c24db6fbba75"
format_stamp: "Formatted at 2026-08-06 22:44:40 on dist-test-slave-h3ft"
server_key: "4cd75429b80cffda151828fb87406e08"
server_key_iv: "7875314836554e622aa04926f5c3601e"
server_key_version: "encryptionkey@0"
I20260806 22:44:40.464126 19873 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.005s sys 0.000s
I20260806 22:44:40.465111 19898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:40.465328 19873 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:44:40.465435 19873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/minicluster-data/ts-2/wal
uuid: "38c55fab9ce14140bcc2c24db6fbba75"
format_stamp: "Formatted at 2026-08-06 22:44:40 on dist-test-slave-h3ft"
server_key: "4cd75429b80cffda151828fb87406e08"
server_key_iv: "7875314836554e622aa04926f5c3601e"
server_key_version: "encryptionkey@0"
I20260806 22:44:40.465530 19873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:40.493216 19873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:40.495934 19873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:40.496140 19873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:40.496477 19873 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:44:40.496891 19873 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:44:40.496949 19873 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:40.496997 19873 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:44:40.497045 19873 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:40.505257 19873 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.195:33387
I20260806 22:44:40.505327 20067 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.195:33387 every 8 connection(s)
I20260806 22:44:40.505750 19873 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
I20260806 22:44:40.509864 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 19873
I20260806 22:44:40.509994 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786056275803658-18615-0/minicluster-data/ts-2/wal/instance
I20260806 22:44:40.510201 18615 external_mini_cluster.cc:1428] Setting key 66fd7e039226d5f03f3202d1ad6a4422
Aug 06 22:44:40 dist-test-slave-h3ft krb5kdc[19350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056280, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.195@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:44:40.517362 19464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:40.506768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.195:56757 (local address 127.18.45.254:44359)
0806 22:44:40.506854 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:40.506856 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:40.507157 (+ 301us) server_negotiation.cc:407] Connection header received
0806 22:44:40.508279 (+ 1122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:40.508281 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:40.508296 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:40.508384 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:40.509171 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:40.510026 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:40.511634 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:40.511832 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:40.511894 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:40.514425 (+ 2531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:40.514426 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:40.514427 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:40.514435 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:40.515991 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:40.516411 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:40.516412 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:40.516412 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:40.516477 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:40.516717 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:40.516718 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:40.516718 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:40.516838 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:40.516903 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:40.517200 (+ 297us) server_negotiation.cc:299] Negotiation successful
0806 22:44:40.517282 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 22:44:40.517542 20072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:40.506718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:44359 (local address 127.18.45.195:56757)
0806 22:44:40.506957 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:40.507046 (+ 89us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:40.508146 (+ 1100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:40.508480 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:40.508484 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:40.508800 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:40.509065 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:40.509070 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:40.510144 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:40.510145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:40.511544 (+ 1399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:40.511549 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:40.511923 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:40.511924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:40.512049 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:40.512715 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:44:40.512725 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:40.514299 (+ 1574us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:40.516074 (+ 1775us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:40.516081 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:40.516083 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:40.516327 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:40.516544 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:40.516545 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:40.516545 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:40.516657 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:40.516897 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:40.516902 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:40.517040 (+ 138us) client_negotiation.cc:769] Sending connection context
0806 22:44:40.517134 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 22:44:40.517280 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":146,"threads_started":1}
I20260806 22:44:40.518123 20069 heartbeater.cc:347] Connected to a master server at 127.18.45.254:44359
I20260806 22:44:40.518229 20069 heartbeater.cc:464] Registering TS with master...
I20260806 22:44:40.518409 20069 heartbeater.cc:511] Master 127.18.45.254:44359 requested a full tablet report, sending...
I20260806 22:44:40.518787 19409 ts_manager.cc:194] Registered new tserver with Master: 38c55fab9ce14140bcc2c24db6fbba75 (127.18.45.195:33387)
I20260806 22:44:40.519258 19409 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.45.195@KRBTEST.COM'} at 127.18.45.195:56757
I20260806 22:44:40.522383 18615 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:44:40.531711 19464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:40.524909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55730 (local address 127.18.45.254:44359)
0806 22:44:40.525008 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:40.525011 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:40.525127 (+ 116us) server_negotiation.cc:407] Connection header received
0806 22:44:40.525228 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:40.525230 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:40.525241 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:40.525316 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:40.526013 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:40.526570 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:40.527424 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:40.527633 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:40.527723 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:40.529095 (+ 1372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:40.529097 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:40.529097 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:40.529107 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:40.530386 (+ 1279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:40.530767 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:40.530768 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:40.530769 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:40.530833 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:40.531084 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:40.531085 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:40.531085 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:40.531264 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:40.531395 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:40.531498 (+ 103us) server_negotiation.cc:299] Negotiation successful
0806 22:44:40.531588 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 22:44:40.534971 19409 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55730:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:44:40.537429 19409 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:44:40.545832 20083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:40.540859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.193:34597 (local address 127.0.0.1:46924)
0806 22:44:40.541327 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:40.541339 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:40.541391 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:40.542254 (+ 863us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:40.542255 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:40.542257 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:44:40.542325 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:40.542327 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:40.543297 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:40.543298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:40.544754 (+ 1456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:40.544756 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:40.545457 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:40.545458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:40.545551 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:40.545567 (+ 16us) client_negotiation.cc:769] Sending connection context
0806 22:44:40.545663 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 22:44:40.545726 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"mutex_wait_us":61,"thread_start_us":116,"threads_started":1}
I20260806 22:44:40.545945 20086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:40.541273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48048 (local address 127.18.45.195:33387)
0806 22:44:40.541530 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:40.541536 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:40.541925 (+ 389us) server_negotiation.cc:407] Connection header received
0806 22:44:40.541982 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:40.541984 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:40.542003 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:40.542255 (+ 252us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:44:40.542357 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:40.543454 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:40.544800 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:40.545563 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:40.545663 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:40.545731 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:40.545796 (+ 65us) server_negotiation.cc:299] Negotiation successful
0806 22:44:40.545845 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":131,"threads_started":1}
I20260806 22:44:40.545961 20082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:40.540856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.194:34659 (local address 127.0.0.1:50984)
0806 22:44:40.541032 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:40.541078 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:40.541202 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:40.541746 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:40.541749 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:40.541752 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:44:40.541826 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:40.541828 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:40.543378 (+ 1550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:40.543379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:40.544416 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:40.544419 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:40.545688 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:40.545689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:40.545752 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:40.545756 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:44:40.545790 (+ 34us) client_negotiation.cc:240] Negotiation successful
0806 22:44:40.545842 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":112,"thread_start_us":86,"threads_started":1}
I20260806 22:44:40.546154 20088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:40.541325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50984 (local address 127.18.45.194:34659)
0806 22:44:40.541568 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:40.541572 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:40.541582 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:44:40.541624 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:40.541626 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:40.541642 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:40.541715 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:44:40.541954 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:40.543221 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:40.544850 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:40.545588 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:40.545892 (+ 304us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:40.545968 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:40.545996 (+ 28us) server_negotiation.cc:299] Negotiation successful
0806 22:44:40.546059 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":108,"threads_started":1}
I20260806 22:44:40.546217 20085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:40.541118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.195:33387 (local address 127.0.0.1:48048)
0806 22:44:40.541845 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:40.541856 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:40.541914 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:40.542126 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:40.542128 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:40.542130 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:44:40.542183 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:40.542184 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:40.543526 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:40.543526 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:40.544727 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:40.544730 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:40.545668 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:40.545669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:40.545725 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:40.545728 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:44:40.545783 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 22:44:40.545819 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":687,"mutex_wait_us":37,"thread_start_us":83,"threads_started":1}
I20260806 22:44:40.546823 20084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:40.541041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46924 (local address 127.18.45.193:34597)
0806 22:44:40.542054 (+ 1013us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:40.542058 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:40.542069 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:44:40.542113 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:40.542116 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:40.542133 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:40.542248 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:44:40.542389 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:40.543175 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:40.544823 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:40.545362 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:40.545458 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:40.545530 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:40.546703 (+ 1173us) server_negotiation.cc:299] Negotiation successful
0806 22:44:40.546752 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":968,"thread_start_us":138,"threads_started":1}
I20260806 22:44:40.547199 19771 tablet_service.cc:1467] Processing CreateTablet for tablet 31b07662a20a4be3825862583ec68c2f (DEFAULT_TABLE table=test-table [id=0265dc0f50d1438db541b6d706e87dbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:44:40.547359 19582 tablet_service.cc:1467] Processing CreateTablet for tablet 31b07662a20a4be3825862583ec68c2f (DEFAULT_TABLE table=test-table [id=0265dc0f50d1438db541b6d706e87dbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:44:40.547677 19582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31b07662a20a4be3825862583ec68c2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:40.547678 19771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31b07662a20a4be3825862583ec68c2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:40.548435 19966 tablet_service.cc:1467] Processing CreateTablet for tablet 31b07662a20a4be3825862583ec68c2f (DEFAULT_TABLE table=test-table [id=0265dc0f50d1438db541b6d706e87dbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:44:40.548808 19966 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31b07662a20a4be3825862583ec68c2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:40.550967 20089 tablet_bootstrap.cc:492] T 31b07662a20a4be3825862583ec68c2f P 1f29d7059482448b9df6fa27758283c3: Bootstrap starting.
I20260806 22:44:40.551589 20090 tablet_bootstrap.cc:492] T 31b07662a20a4be3825862583ec68c2f P a079331ebcb141fb86b9fe54245bb2b3: Bootstrap starting.
I20260806 22:44:40.552084 20089 tablet_bootstrap.cc:654] T 31b07662a20a4be3825862583ec68c2f P 1f29d7059482448b9df6fa27758283c3: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:40.552444 20092 tablet_bootstrap.cc:492] T 31b07662a20a4be3825862583ec68c2f P 38c55fab9ce14140bcc2c24db6fbba75: Bootstrap starting.
I20260806 22:44:40.552518 20090 tablet_bootstrap.cc:654] T 31b07662a20a4be3825862583ec68c2f P a079331ebcb141fb86b9fe54245bb2b3: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:40.553084 20089 log.cc:824] T 31b07662a20a4be3825862583ec68c2f P 1f29d7059482448b9df6fa27758283c3: Log is configured to *not* fsync() on all Append() calls
I20260806 22:44:40.553221 20090 log.cc:824] T 31b07662a20a4be3825862583ec68c2f P a079331ebcb141fb86b9fe54245bb2b3: Log is configured to *not* fsync() on all Append() calls
I20260806 22:44:40.553344 20092 tablet_bootstrap.cc:654] T 31b07662a20a4be3825862583ec68c2f P 38c55fab9ce14140bcc2c24db6fbba75: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:40.553992 20089 tablet_bootstrap.cc:492] T 31b07662a20a4be3825862583ec68c2f P 1f29d7059482448b9df6fa27758283c3: No bootstrap required, opened a new log
I20260806 22:44:40.554028 20092 log.cc:824] T 31b07662a20a4be3825862583ec68c2f P 38c55fab9ce14140bcc2c24db6fbba75: Log is configured to *not* fsync() on all Append() calls
I20260806 22:44:40.554088 20089 ts_tablet_manager.cc:1397] T 31b07662a20a4be3825862583ec68c2f P 1f29d7059482448b9df6fa27758283c3: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260806 22:44:40.554117 20090 tablet_bootstrap.cc:492] T 31b07662a20a4be3825862583ec68c2f P a079331ebcb141fb86b9fe54245bb2b3: No bootstrap required, opened a new log
I20260806 22:44:40.554178 20090 ts_tablet_manager.cc:1397] T 31b07662a20a4be3825862583ec68c2f P a079331ebcb141fb86b9fe54245bb2b3: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:44:40.554873 20092 tablet_bootstrap.cc:492] T 31b07662a20a4be3825862583ec68c2f P 38c55fab9ce14140bcc2c24db6fbba75: No bootstrap required, opened a new log
I20260806 22:44:40.554949 20092 ts_tablet_manager.cc:1397] T 31b07662a20a4be3825862583ec68c2f P 38c55fab9ce14140bcc2c24db6fbba75: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:44:40.556124 20089 raft_consensus.cc:348] T 31b07662a20a4be3825862583ec68c2f P 1f29d7059482448b9df6fa27758283c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f29d7059482448b9df6fa27758283c3" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 34659 } } peers { permanent_uuid: "38c55fab9ce14140bcc2c24db6fbba75" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 33387 } } peers { permanent_uuid: "a079331ebcb141fb86b9fe54245bb2b3" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 34597 } }
I20260806 22:44:40.556310 20089 raft_consensus.cc:374] T 31b07662a20a4be3825862583ec68c2f P 1f29d7059482448b9df6fa27758283c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:40.556360 20089 raft_consensus.cc:729] T 31b07662a20a4be3825862583ec68c2f P 1f29d7059482448b9df6fa27758283c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f29d7059482448b9df6fa27758283c3, State: Initialized, Role: FOLLOWER
I20260806 22:44:40.556535 20089 consensus_queue.cc:260] T 31b07662a20a4be3825862583ec68c2f P 1f29d7059482448b9df6fa27758283c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f29d7059482448b9df6fa27758283c3" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 34659 } } peers { permanent_uuid: "38c55fab9ce14140bcc2c24db6fbba75" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 33387 } } peers { permanent_uuid: "a079331ebcb141fb86b9fe54245bb2b3" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 34597 } }
I20260806 22:44:40.556917 20089 ts_tablet_manager.cc:1428] T 31b07662a20a4be3825862583ec68c2f P 1f29d7059482448b9df6fa27758283c3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:44:40.556937 19867 heartbeater.cc:503] Master 127.18.45.254:44359 was elected leader, sending a full tablet report...
I20260806 22:44:40.556892 20090 raft_consensus.cc:348] T 31b07662a20a4be3825862583ec68c2f P a079331ebcb141fb86b9fe54245bb2b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f29d7059482448b9df6fa27758283c3" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 34659 } } peers { permanent_uuid: "38c55fab9ce14140bcc2c24db6fbba75" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 33387 } } peers { permanent_uuid: "a079331ebcb141fb86b9fe54245bb2b3" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 34597 } }
I20260806 22:44:40.557081 20090 raft_consensus.cc:374] T 31b07662a20a4be3825862583ec68c2f P a079331ebcb141fb86b9fe54245bb2b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:40.557121 20090 raft_consensus.cc:729] T 31b07662a20a4be3825862583ec68c2f P a079331ebcb141fb86b9fe54245bb2b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a079331ebcb141fb86b9fe54245bb2b3, State: Initialized, Role: FOLLOWER
I20260806 22:44:40.557248 20090 consensus_queue.cc:260] T 31b07662a20a4be3825862583ec68c2f P a079331ebcb141fb86b9fe54245bb2b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f29d7059482448b9df6fa27758283c3" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 34659 } } peers { permanent_uuid: "38c55fab9ce14140bcc2c24db6fbba75" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 33387 } } peers { permanent_uuid: "a079331ebcb141fb86b9fe54245bb2b3" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 34597 } }
I20260806 22:44:40.557546 20090 ts_tablet_manager.cc:1428] T 31b07662a20a4be3825862583ec68c2f P a079331ebcb141fb86b9fe54245bb2b3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:44:40.557565 20092 raft_consensus.cc:348] T 31b07662a20a4be3825862583ec68c2f P 38c55fab9ce14140bcc2c24db6fbba75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f29d7059482448b9df6fa27758283c3" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 34659 } } peers { permanent_uuid: "38c55fab9ce14140bcc2c24db6fbba75" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 33387 } } peers { permanent_uuid: "a079331ebcb141fb86b9fe54245bb2b3" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 34597 } }
I20260806 22:44:40.557714 19673 heartbeater.cc:503] Master 127.18.45.254:44359 was elected leader, sending a full tablet report...
I20260806 22:44:40.557734 20092 raft_consensus.cc:374] T 31b07662a20a4be3825862583ec68c2f P 38c55fab9ce14140bcc2c24db6fbba75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:40.557776 20092 raft_consensus.cc:729] T 31b07662a20a4be3825862583ec68c2f P 38c55fab9ce14140bcc2c24db6fbba75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38c55fab9ce14140bcc2c24db6fbba75, State: Initialized, Role: FOLLOWER
I20260806 22:44:40.557888 20092 consensus_queue.cc:260] T 31b07662a20a4be3825862583ec68c2f P 38c55fab9ce14140bcc2c24db6fbba75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f29d7059482448b9df6fa27758283c3" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 34659 } } peers { permanent_uuid: "38c55fab9ce14140bcc2c24db6fbba75" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 33387 } } peers { permanent_uuid: "a079331ebcb141fb86b9fe54245bb2b3" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 34597 } }
I20260806 22:44:40.558175 20092 ts_tablet_manager.cc:1428] T 31b07662a20a4be3825862583ec68c2f P 38c55fab9ce14140bcc2c24db6fbba75: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260806 22:44:40.558332 20069 heartbeater.cc:503] Master 127.18.45.254:44359 was elected leader, sending a full tablet report...
W20260806 22:44:40.604779 19674 tablet.cc:2369] T 31b07662a20a4be3825862583ec68c2f P a079331ebcb141fb86b9fe54245bb2b3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:44:40.663229 20096 raft_consensus.cc:482] T 31b07662a20a4be3825862583ec68c2f P 1f29d7059482448b9df6fa27758283c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:44:40.663410 20096 raft_consensus.cc:504] T 31b07662a20a4be3825862583ec68c2f P 1f29d7059482448b9df6fa27758283c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f29d7059482448b9df6fa27758283c3" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 34659 } } peers { permanent_uuid: "38c55fab9ce14140bcc2c24db6fbba75" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 33387 } } peers { permanent_uuid: "a079331ebcb141fb86b9fe54245bb2b3" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 34597 } }
I20260806 22:44:40.664285 20096 leader_election.cc:288] T 31b07662a20a4be3825862583ec68c2f P 1f29d7059482448b9df6fa27758283c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38c55fab9ce14140bcc2c24db6fbba75 (127.18.45.195:33387), a079331ebcb141fb86b9fe54245bb2b3 (127.18.45.193:34597)
I20260806 22:44:40.668817 20084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:40.664703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:53503 (local address 127.18.45.193:34597)
0806 22:44:40.664786 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:40.664789 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:40.664800 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:44:40.664851 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:40.664852 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:40.664862 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:40.664912 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:44:40.665225 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:40.665980 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:40.667745 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:40.668374 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:40.668465 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:40.668477 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:40.668694 (+ 217us) server_negotiation.cc:299] Negotiation successful
0806 22:44:40.668735 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 22:44:40.668817 20086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:40.664547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:49881 (local address 127.18.45.195:33387)
0806 22:44:40.664630 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:40.664633 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:40.664662 (+ 29us) server_negotiation.cc:407] Connection header received
0806 22:44:40.664704 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:40.664705 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:40.664715 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:40.664788 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:44:40.665036 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:40.665978 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:40.667746 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:40.668374 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:40.668467 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:40.668477 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:40.668669 (+ 192us) server_negotiation.cc:299] Negotiation successful
0806 22:44:40.668726 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 22:44:40.668839 19869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:40.664453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.195:33387 (local address 127.18.45.194:49881)
0806 22:44:40.664526 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:40.664540 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:40.664624 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:40.664796 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:40.664798 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:40.664800 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:44:40.664928 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:40.664931 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:40.666090 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:40.666091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:40.667645 (+ 1554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:40.667649 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:40.668478 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:40.668479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:40.668589 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:40.668592 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:44:40.668660 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 22:44:40.668695 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":108}
I20260806 22:44:40.668848 20101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:40.664466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.193:34597 (local address 127.18.45.194:53503)
0806 22:44:40.664722 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:40.664733 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:40.664795 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:40.664932 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:40.664932 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:40.664934 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:44:40.665033 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:40.665034 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:40.666090 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:40.666091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:40.667645 (+ 1554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:40.667649 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:40.668478 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:40.668479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:40.668565 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:40.668572 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 22:44:40.668633 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 22:44:40.668688 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"mutex_wait_us":89,"thread_start_us":159,"threads_started":1}
I20260806 22:44:40.669436 19998 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31b07662a20a4be3825862583ec68c2f" candidate_uuid: "1f29d7059482448b9df6fa27758283c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c55fab9ce14140bcc2c24db6fbba75" is_pre_election: true
I20260806 22:44:40.669541 19611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31b07662a20a4be3825862583ec68c2f" candidate_uuid: "1f29d7059482448b9df6fa27758283c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a079331ebcb141fb86b9fe54245bb2b3" is_pre_election: true
I20260806 22:44:40.669729 19611 raft_consensus.cc:2454] T 31b07662a20a4be3825862583ec68c2f P a079331ebcb141fb86b9fe54245bb2b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f29d7059482448b9df6fa27758283c3 in term 0.
I20260806 22:44:40.669729 19998 raft_consensus.cc:2454] T 31b07662a20a4be3825862583ec68c2f P 38c55fab9ce14140bcc2c24db6fbba75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f29d7059482448b9df6fa27758283c3 in term 0.
I20260806 22:44:40.670081 19702 leader_election.cc:302] T 31b07662a20a4be3825862583ec68c2f P 1f29d7059482448b9df6fa27758283c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f29d7059482448b9df6fa27758283c3, a079331ebcb141fb86b9fe54245bb2b3; no voters:
I20260806 22:44:40.670363 20096 raft_consensus.cc:2781] T 31b07662a20a4be3825862583ec68c2f P 1f29d7059482448b9df6fa27758283c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:44:40.670413 20096 raft_consensus.cc:482] T 31b07662a20a4be3825862583ec68c2f P 1f29d7059482448b9df6fa27758283c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:44:40.670429 20096 raft_consensus.cc:3037] T 31b07662a20a4be3825862583ec68c2f P 1f29d7059482448b9df6fa27758283c3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:40.671504 20096 raft_consensus.cc:504] T 31b07662a20a4be3825862583ec68c2f P 1f29d7059482448b9df6fa27758283c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f29d7059482448b9df6fa27758283c3" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 34659 } } peers { permanent_uuid: "38c55fab9ce14140bcc2c24db6fbba75" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 33387 } } peers { permanent_uuid: "a079331ebcb141fb86b9fe54245bb2b3" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 34597 } }
I20260806 22:44:40.671671 20096 leader_election.cc:288] T 31b07662a20a4be3825862583ec68c2f P 1f29d7059482448b9df6fa27758283c3 [CANDIDATE]: Term 1 election: Requested vote from peers 38c55fab9ce14140bcc2c24db6fbba75 (127.18.45.195:33387), a079331ebcb141fb86b9fe54245bb2b3 (127.18.45.193:34597)
I20260806 22:44:40.671902 19998 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31b07662a20a4be3825862583ec68c2f" candidate_uuid: "1f29d7059482448b9df6fa27758283c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38c55fab9ce14140bcc2c24db6fbba75"
I20260806 22:44:40.671903 19611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31b07662a20a4be3825862583ec68c2f" candidate_uuid: "1f29d7059482448b9df6fa27758283c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a079331ebcb141fb86b9fe54245bb2b3"
I20260806 22:44:40.672000 19611 raft_consensus.cc:3037] T 31b07662a20a4be3825862583ec68c2f P a079331ebcb141fb86b9fe54245bb2b3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:40.671996 19998 raft_consensus.cc:3037] T 31b07662a20a4be3825862583ec68c2f P 38c55fab9ce14140bcc2c24db6fbba75 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:40.673127 19611 raft_consensus.cc:2454] T 31b07662a20a4be3825862583ec68c2f P a079331ebcb141fb86b9fe54245bb2b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f29d7059482448b9df6fa27758283c3 in term 1.
I20260806 22:44:40.673225 19998 raft_consensus.cc:2454] T 31b07662a20a4be3825862583ec68c2f P 38c55fab9ce14140bcc2c24db6fbba75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f29d7059482448b9df6fa27758283c3 in term 1.
I20260806 22:44:40.673398 19702 leader_election.cc:302] T 31b07662a20a4be3825862583ec68c2f P 1f29d7059482448b9df6fa27758283c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f29d7059482448b9df6fa27758283c3, a079331ebcb141fb86b9fe54245bb2b3; no voters:
I20260806 22:44:40.673540 20096 raft_consensus.cc:2781] T 31b07662a20a4be3825862583ec68c2f P 1f29d7059482448b9df6fa27758283c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:44:40.673764 20096 raft_consensus.cc:686] T 31b07662a20a4be3825862583ec68c2f P 1f29d7059482448b9df6fa27758283c3 [term 1 LEADER]: Becoming Leader. State: Replica: 1f29d7059482448b9df6fa27758283c3, State: Running, Role: LEADER
I20260806 22:44:40.673887 20096 consensus_queue.cc:237] T 31b07662a20a4be3825862583ec68c2f P 1f29d7059482448b9df6fa27758283c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f29d7059482448b9df6fa27758283c3" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 34659 } } peers { permanent_uuid: "38c55fab9ce14140bcc2c24db6fbba75" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 33387 } } peers { permanent_uuid: "a079331ebcb141fb86b9fe54245bb2b3" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 34597 } }
I20260806 22:44:40.675061 19408 catalog_manager.cc:5495] T 31b07662a20a4be3825862583ec68c2f P 1f29d7059482448b9df6fa27758283c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f29d7059482448b9df6fa27758283c3 (127.18.45.194). New cstate: current_term: 1 leader_uuid: "1f29d7059482448b9df6fa27758283c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f29d7059482448b9df6fa27758283c3" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 34659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38c55fab9ce14140bcc2c24db6fbba75" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 33387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a079331ebcb141fb86b9fe54245bb2b3" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 34597 } health_report { overall_health: UNKNOWN } } }
Aug 06 22:44:40 dist-test-slave-h3ft krb5kdc[19350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056279, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.45.193@KRBTEST.COM
I20260806 22:44:40.703408 20084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:40.694531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46940 (local address 127.18.45.193:34597)
0806 22:44:40.694616 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:40.694619 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:40.694764 (+ 145us) server_negotiation.cc:407] Connection header received
0806 22:44:40.694825 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:40.694826 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:40.694838 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:40.694893 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:40.695670 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:40.696225 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:40.696992 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:40.697188 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:40.697246 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:40.700142 (+ 2896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:40.700157 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:40.700157 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:40.700166 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:40.702021 (+ 1855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:40.702452 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:40.702456 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:40.702456 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:40.702540 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:40.702728 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:40.702729 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:40.702729 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:40.702883 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:40.703085 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:40.703199 (+ 114us) server_negotiation.cc:299] Negotiation successful
0806 22:44:40.703277 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
Aug 06 22:44:40 dist-test-slave-h3ft krb5kdc[19350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056280, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 22:44:40 dist-test-slave-h3ft krb5kdc[19350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056280, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.45.193@KRBTEST.COM
I20260806 22:44:40.732592 20084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:40.724344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46954 (local address 127.18.45.193:34597)
0806 22:44:40.724435 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:40.724439 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:40.724569 (+ 130us) server_negotiation.cc:407] Connection header received
0806 22:44:40.724673 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:40.724674 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:40.724691 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:40.724761 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:40.725419 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:40.725964 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:40.726747 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:40.726963 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:40.727102 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:40.729736 (+ 2634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:40.729738 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:40.729738 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:40.729747 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:40.731273 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:40.731645 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:40.731647 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:40.731648 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:40.731707 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:40.731912 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:40.731913 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:40.731914 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:40.732033 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:40.732145 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:40.732360 (+ 215us) server_negotiation.cc:299] Negotiation successful
0806 22:44:40.732449 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
Aug 06 22:44:40 dist-test-slave-h3ft krb5kdc[19350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056280, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
W20260806 22:44:40.756561 20070 tablet.cc:2369] T 31b07662a20a4be3825862583ec68c2f P 38c55fab9ce14140bcc2c24db6fbba75: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 22:44:40 dist-test-slave-h3ft krb5kdc[19350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056280, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.45.193@KRBTEST.COM
I20260806 22:44:40.760121 20084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:40.752198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46968 (local address 127.18.45.193:34597)
0806 22:44:40.752293 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:40.752296 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:40.752401 (+ 105us) server_negotiation.cc:407] Connection header received
0806 22:44:40.752509 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:40.752510 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:40.752523 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:40.752592 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:40.753258 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:40.753797 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:40.754576 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:40.754793 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:40.754891 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:40.757574 (+ 2683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:40.757576 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:40.757576 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:40.757585 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:40.759063 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:40.759385 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:40.759386 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:40.759386 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:40.759446 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:40.759618 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:40.759619 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:40.759619 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:40.759729 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:40.759813 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:40.759913 (+ 100us) server_negotiation.cc:299] Negotiation successful
0806 22:44:40.759981 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260806 22:44:40.760933 19554 tablet_service.cc:2908] Rejecting scan request for tablet 31b07662a20a4be3825862583ec68c2f: Uninitialized: clean time has not yet been initialized
I20260806 22:44:40.762014 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 19491
I20260806 22:44:40.778066 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 19681
I20260806 22:44:40.789747 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 19873
I20260806 22:44:40.800197 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 19368
2026-08-06T22:44:40Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2497 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:44:40 dist-test-slave-h3ft krb5kdc[20125](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:44:40 dist-test-slave-h3ft krb5kdc[20125](info): set up 2 sockets
Aug 06 22:44:40 dist-test-slave-h3ft krb5kdc[20125](info): commencing operation
krb5kdc: starting...
W20260806 22:44:42.059667 18615 mini_kdc.cc:121] Time spent starting KDC: real 1.227s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Aug 06 22:44:42 dist-test-slave-h3ft krb5kdc[20125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056282, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:44:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:44:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.45.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab.
Entry for principal kudu/127.18.45.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab.
I20260806 22:44:42.231243 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:42241
--webserver_interface=127.18.45.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:40553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.45.254:42241
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:44:42.330039 20143 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:42.330350 20143 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:42.330382 20143 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:42.332075 20143 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:44:42.332149 20143 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 22:44:42.332163 20143 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:42.332177 20143 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:42.332190 20143 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:44:42.332203 20143 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:44:42.334290 20143 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:40553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.45.254:42241
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:42241
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.45.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20143
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:44:42.334656 20143 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:42.334976 20143 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:42.341164 20152 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:42.341261 20154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:42.341235 20151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:42.341624 20143 server_base.cc:1034] running on GCE node
I20260806 22:44:42.342011 20143 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:42.342413 20143 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:42.343628 20143 hybrid_clock.cc:648] HybridClock initialized: now 1786056282343605 us; error 43 us; skew 500 ppm
Aug 06 22:44:42 dist-test-slave-h3ft krb5kdc[20125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056282, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:42.351101 20143 init.cc:377] Logged in from keytab as kudu/127.18.45.254@KRBTEST.COM (short username kudu)
I20260806 22:44:42.351773 20143 webserver.cc:460] Webserver started at http://127.18.45.254:34701/ using document root <none> and password file <none>
I20260806 22:44:42.352111 20143 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:42.352192 20143 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:42.352375 20143 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:44:42.353796 20143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/master-0/data/instance:
uuid: "ece890efc8674a71b7ee1a70429fc1bb"
format_stamp: "Formatted at 2026-08-06 22:44:42 on dist-test-slave-h3ft"
server_key: "061a8843c9f31d5592079e502a23b10d"
server_key_iv: "127ad573134a93ca531ce99ac94bc08f"
server_key_version: "encryptionkey@0"
I20260806 22:44:42.354233 20143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/master-0/wal/instance:
uuid: "ece890efc8674a71b7ee1a70429fc1bb"
format_stamp: "Formatted at 2026-08-06 22:44:42 on dist-test-slave-h3ft"
server_key: "061a8843c9f31d5592079e502a23b10d"
server_key_iv: "127ad573134a93ca531ce99ac94bc08f"
server_key_version: "encryptionkey@0"
I20260806 22:44:42.356120 20143 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260806 22:44:42.357117 20166 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:42.357478 20143 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:44:42.357623 20143 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/master-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/master-0/wal
uuid: "ece890efc8674a71b7ee1a70429fc1bb"
format_stamp: "Formatted at 2026-08-06 22:44:42 on dist-test-slave-h3ft"
server_key: "061a8843c9f31d5592079e502a23b10d"
server_key_iv: "127ad573134a93ca531ce99ac94bc08f"
server_key_version: "encryptionkey@0"
I20260806 22:44:42.357721 20143 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:42.374584 20143 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:42.377358 20143 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:42.377566 20143 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:42.384231 20143 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.254:42241
I20260806 22:44:42.384295 20257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.254:42241 every 8 connection(s)
I20260806 22:44:42.384799 20143 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
I20260806 22:44:42.386026 20258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:42.389011 20258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ece890efc8674a71b7ee1a70429fc1bb: Bootstrap starting.
I20260806 22:44:42.389362 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 20143
I20260806 22:44:42.389463 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/master-0/wal/instance
I20260806 22:44:42.389670 18615 external_mini_cluster.cc:1428] Setting key 2c30a269e3d9377fb82db47a00099b27
I20260806 22:44:42.389959 20258 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ece890efc8674a71b7ee1a70429fc1bb: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:42.390381 20258 log.cc:824] T 00000000000000000000000000000000 P ece890efc8674a71b7ee1a70429fc1bb: Log is configured to *not* fsync() on all Append() calls
I20260806 22:44:42.391342 20258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ece890efc8674a71b7ee1a70429fc1bb: No bootstrap required, opened a new log
I20260806 22:44:42.393453 20258 raft_consensus.cc:348] T 00000000000000000000000000000000 P ece890efc8674a71b7ee1a70429fc1bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ece890efc8674a71b7ee1a70429fc1bb" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 42241 } }
I20260806 22:44:42.393671 20258 raft_consensus.cc:374] T 00000000000000000000000000000000 P ece890efc8674a71b7ee1a70429fc1bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:42.393710 20258 raft_consensus.cc:729] T 00000000000000000000000000000000 P ece890efc8674a71b7ee1a70429fc1bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ece890efc8674a71b7ee1a70429fc1bb, State: Initialized, Role: FOLLOWER
I20260806 22:44:42.393862 20258 consensus_queue.cc:260] T 00000000000000000000000000000000 P ece890efc8674a71b7ee1a70429fc1bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ece890efc8674a71b7ee1a70429fc1bb" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 42241 } }
I20260806 22:44:42.393955 20258 raft_consensus.cc:388] T 00000000000000000000000000000000 P ece890efc8674a71b7ee1a70429fc1bb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:44:42.393989 20258 raft_consensus.cc:482] T 00000000000000000000000000000000 P ece890efc8674a71b7ee1a70429fc1bb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:44:42.394044 20258 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ece890efc8674a71b7ee1a70429fc1bb [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:42.394871 20258 raft_consensus.cc:504] T 00000000000000000000000000000000 P ece890efc8674a71b7ee1a70429fc1bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ece890efc8674a71b7ee1a70429fc1bb" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 42241 } }
I20260806 22:44:42.394987 20258 leader_election.cc:302] T 00000000000000000000000000000000 P ece890efc8674a71b7ee1a70429fc1bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ece890efc8674a71b7ee1a70429fc1bb; no voters:
I20260806 22:44:42.395256 20258 leader_election.cc:288] T 00000000000000000000000000000000 P ece890efc8674a71b7ee1a70429fc1bb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:44:42.395315 20264 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ece890efc8674a71b7ee1a70429fc1bb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:44:42.395517 20264 raft_consensus.cc:686] T 00000000000000000000000000000000 P ece890efc8674a71b7ee1a70429fc1bb [term 1 LEADER]: Becoming Leader. State: Replica: ece890efc8674a71b7ee1a70429fc1bb, State: Running, Role: LEADER
I20260806 22:44:42.395622 20264 consensus_queue.cc:237] T 00000000000000000000000000000000 P ece890efc8674a71b7ee1a70429fc1bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ece890efc8674a71b7ee1a70429fc1bb" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 42241 } }
I20260806 22:44:42.395648 20258 sys_catalog.cc:564] T 00000000000000000000000000000000 P ece890efc8674a71b7ee1a70429fc1bb [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:44:42.397502 20265 sys_catalog.cc:455] T 00000000000000000000000000000000 P ece890efc8674a71b7ee1a70429fc1bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader ece890efc8674a71b7ee1a70429fc1bb. Latest consensus state: current_term: 1 leader_uuid: "ece890efc8674a71b7ee1a70429fc1bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ece890efc8674a71b7ee1a70429fc1bb" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 42241 } } }
I20260806 22:44:42.397603 20265 sys_catalog.cc:458] T 00000000000000000000000000000000 P ece890efc8674a71b7ee1a70429fc1bb [sys.catalog]: This master's current role is: LEADER
I20260806 22:44:42.397780 20266 sys_catalog.cc:455] T 00000000000000000000000000000000 P ece890efc8674a71b7ee1a70429fc1bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ece890efc8674a71b7ee1a70429fc1bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ece890efc8674a71b7ee1a70429fc1bb" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 42241 } } }
I20260806 22:44:42.397987 20266 sys_catalog.cc:458] T 00000000000000000000000000000000 P ece890efc8674a71b7ee1a70429fc1bb [sys.catalog]: This master's current role is: LEADER
W20260806 22:44:42.398041 20283 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ece890efc8674a71b7ee1a70429fc1bb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:44:42.398095 20283 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:44:42.398192 20285 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:44:42.398468 20285 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 22:44:42 dist-test-slave-h3ft krb5kdc[20125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056282, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:44:42.400520 20285 catalog_manager.cc:1347] Generated new cluster ID: 1855f8ae465c4680906a9afc1e5448b7
I20260806 22:44:42.400594 20285 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:44:42.405197 20262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:42.390371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37632 (local address 127.18.45.254:42241)
0806 22:44:42.390632 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:42.390644 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:42.390690 (+ 46us) server_negotiation.cc:407] Connection header received
0806 22:44:42.391449 (+ 759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:42.391460 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:42.391545 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:42.391729 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:42.392539 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:42.394561 (+ 2022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:42.395857 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:42.396559 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:42.396678 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:42.400349 (+ 3671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:42.400354 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:42.400354 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:42.400370 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:42.402621 (+ 2251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:42.403143 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:42.403148 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:42.403148 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:42.403275 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:42.403507 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:42.403508 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:42.403508 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:42.403884 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:42.404070 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:42.404718 (+ 648us) server_negotiation.cc:299] Negotiation successful
0806 22:44:42.404907 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":99,"threads_started":1}
I20260806 22:44:42.408919 20285 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:44:42.409669 20285 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:44:42.413233 20285 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ece890efc8674a71b7ee1a70429fc1bb: Generated new TSK 0
I20260806 22:44:42.413456 20285 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.45.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab.
Entry for principal kudu/127.18.45.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab.
I20260806 22:44:42.452618 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.193:0
--local_ip_for_outbound_sockets=127.18.45.193
--webserver_interface=127.18.45.193
--webserver_port=0
--tserver_master_addrs=127.18.45.254:42241
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab
--principal=kudu/127.18.45.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:40553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:44:42.551208 20290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:42.551470 20290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:42.551499 20290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:42.552760 20290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:42.552801 20290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:42.552822 20290 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:44:42.552841 20290 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.193
I20260806 22:44:42.554414 20290 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:40553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab
--principal=kudu/127.18.45.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.45.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.45.254:42241
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20290
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.193
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:44:42.554669 20290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:42.554915 20290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:42.560949 20298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:42.560949 20303 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:42.561079 20299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:42.561245 20290 server_base.cc:1034] running on GCE node
I20260806 22:44:42.561476 20290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:42.561785 20290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:42.562983 20290 hybrid_clock.cc:648] HybridClock initialized: now 1786056282562969 us; error 45 us; skew 500 ppm
Aug 06 22:44:42 dist-test-slave-h3ft krb5kdc[20125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056282, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:42.566226 20290 init.cc:377] Logged in from keytab as kudu/127.18.45.193@KRBTEST.COM (short username kudu)
I20260806 22:44:42.566741 20290 webserver.cc:460] Webserver started at http://127.18.45.193:32821/ using document root <none> and password file <none>
I20260806 22:44:42.567052 20290 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:42.567123 20290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:42.567239 20290 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:44:42.568464 20290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/ts-0/data/instance:
uuid: "68a1a25643ff4b9aad520e2d7a1f5008"
format_stamp: "Formatted at 2026-08-06 22:44:42 on dist-test-slave-h3ft"
server_key: "7b550f10061d3353d35025c52cd00c46"
server_key_iv: "4475c4a08dc73a78c8f26e6539f46ce1"
server_key_version: "encryptionkey@0"
I20260806 22:44:42.568849 20290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/ts-0/wal/instance:
uuid: "68a1a25643ff4b9aad520e2d7a1f5008"
format_stamp: "Formatted at 2026-08-06 22:44:42 on dist-test-slave-h3ft"
server_key: "7b550f10061d3353d35025c52cd00c46"
server_key_iv: "4475c4a08dc73a78c8f26e6539f46ce1"
server_key_version: "encryptionkey@0"
I20260806 22:44:42.570451 20290 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 22:44:42.571435 20316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:42.571641 20290 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:44:42.571774 20290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/ts-0/wal
uuid: "68a1a25643ff4b9aad520e2d7a1f5008"
format_stamp: "Formatted at 2026-08-06 22:44:42 on dist-test-slave-h3ft"
server_key: "7b550f10061d3353d35025c52cd00c46"
server_key_iv: "4475c4a08dc73a78c8f26e6539f46ce1"
server_key_version: "encryptionkey@0"
I20260806 22:44:42.571873 20290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:42.580157 20290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:42.582934 20290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:42.583153 20290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:42.583775 20290 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:44:42.583846 20290 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:42.583886 20290 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:44:42.583930 20290 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 22:44:42 dist-test-slave-h3ft krb5kdc[20125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056282, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.193@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:44:42.593372 20290 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.193:33651
I20260806 22:44:42.593444 20479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.193:33651 every 8 connection(s)
I20260806 22:44:42.593935 20290 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
I20260806 22:44:42.596659 20262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:42.584501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:58509 (local address 127.18.45.254:42241)
0806 22:44:42.584604 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:42.584607 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:42.584836 (+ 229us) server_negotiation.cc:407] Connection header received
0806 22:44:42.585497 (+ 661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:42.585498 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:42.585514 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:42.585570 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:42.586592 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:42.587644 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:42.588966 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:42.589391 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:42.589494 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:42.593014 (+ 3520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:42.593017 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:42.593017 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:42.593028 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:42.595057 (+ 2029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:42.595468 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:42.595470 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:42.595470 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:42.595539 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:42.595840 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:42.595841 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:42.595842 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:42.596007 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:42.596070 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:42.596417 (+ 347us) server_negotiation.cc:299] Negotiation successful
0806 22:44:42.596575 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 22:44:42.596745 20328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:42.584445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:42241 (local address 127.18.45.193:58509)
0806 22:44:42.584680 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:42.584727 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:42.585366 (+ 639us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:42.585649 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:42.585654 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:42.586081 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:42.586442 (+ 361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:42.586450 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:42.587797 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:42.587798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:42.588846 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:42.588850 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:42.589513 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:42.589514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:42.589671 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:42.590326 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:44:42.590338 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:42.592887 (+ 2549us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:42.595145 (+ 2258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:42.595150 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:42.595152 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:42.595385 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:42.595607 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:42.595607 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:42.595608 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:42.595714 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:42.596082 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:42.596088 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:42.596221 (+ 133us) client_negotiation.cc:769] Sending connection context
0806 22:44:42.596311 (+ 90us) client_negotiation.cc:240] Negotiation successful
0806 22:44:42.596553 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":117,"threads_started":1}
I20260806 22:44:42.597401 20481 heartbeater.cc:347] Connected to a master server at 127.18.45.254:42241
I20260806 22:44:42.597533 20481 heartbeater.cc:464] Registering TS with master...
I20260806 22:44:42.597770 20481 heartbeater.cc:511] Master 127.18.45.254:42241 requested a full tablet report, sending...
I20260806 22:44:42.598507 20189 ts_manager.cc:194] Registered new tserver with Master: 68a1a25643ff4b9aad520e2d7a1f5008 (127.18.45.193:33651)
I20260806 22:44:42.599596 20189 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.45.193@KRBTEST.COM'} at 127.18.45.193:58509
I20260806 22:44:42.600369 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 20290
I20260806 22:44:42.600500 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/ts-0/wal/instance
I20260806 22:44:42.600731 18615 external_mini_cluster.cc:1428] Setting key 517f253a2c371979f97a0fef06fa266c
I20260806 22:44:42.605814 20262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:42.598702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:34295 (local address 127.18.45.254:42241)
0806 22:44:42.598753 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:42.598755 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:42.598940 (+ 185us) server_negotiation.cc:407] Connection header received
0806 22:44:42.598964 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:42.598965 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:42.598978 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:42.599076 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:42.599395 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:42.600167 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:42.601276 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:42.601604 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:42.601690 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:42.602825 (+ 1135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:42.602827 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:42.602827 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:42.602834 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:42.604479 (+ 1645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:42.604888 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:42.604890 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:42.604891 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:42.604967 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:42.605195 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:42.605196 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:42.605197 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:42.605330 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:42.605413 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:42.605616 (+ 203us) server_negotiation.cc:299] Negotiation successful
0806 22:44:42.605702 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 22:44:42.605826 20489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:42.598650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:42241 (local address 127.18.45.193:34295)
0806 22:44:42.598822 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:42.598840 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:42.598899 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:42.599082 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:42.599083 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:42.599251 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:42.599321 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:42.599322 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:42.600293 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:42.600294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:42.601172 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:42.601174 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:42.601687 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:42.601688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:42.601782 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:42.602332 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:44:42.602337 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:42.602705 (+ 368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:42.604591 (+ 1886us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:42.604592 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:42.604593 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:42.604790 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:42.605043 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:42.605044 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:42.605045 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:42.605121 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:42.605421 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:42.605423 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:42.605499 (+ 76us) client_negotiation.cc:769] Sending connection context
0806 22:44:42.605583 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 22:44:42.605707 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":99,"threads_started":1}
WARNING: no policy specified for kudu/127.18.45.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab.
Entry for principal kudu/127.18.45.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab.
I20260806 22:44:42.634379 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.194:0
--local_ip_for_outbound_sockets=127.18.45.194
--webserver_interface=127.18.45.194
--webserver_port=0
--tserver_master_addrs=127.18.45.254:42241
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab
--principal=kudu/127.18.45.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:40553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:44:42.736229 20492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:42.736567 20492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:42.736619 20492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:42.738091 20492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:42.738173 20492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:42.738209 20492 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:44:42.738255 20492 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.194
I20260806 22:44:42.740053 20492 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:40553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab
--principal=kudu/127.18.45.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.45.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.45.254:42241
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20492
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.194
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:44:42.740406 20492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:42.740708 20492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:42.747228 20502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:42.747228 20500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:42.747256 20492 server_base.cc:1034] running on GCE node
W20260806 22:44:42.747234 20504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:42.747936 20492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:42.748291 20492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:42.749491 20492 hybrid_clock.cc:648] HybridClock initialized: now 1786056282749481 us; error 41 us; skew 500 ppm
Aug 06 22:44:42 dist-test-slave-h3ft krb5kdc[20125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056282, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:42.753115 20492 init.cc:377] Logged in from keytab as kudu/127.18.45.194@KRBTEST.COM (short username kudu)
I20260806 22:44:42.753700 20492 webserver.cc:460] Webserver started at http://127.18.45.194:45623/ using document root <none> and password file <none>
I20260806 22:44:42.754035 20492 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:42.754118 20492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:42.754302 20492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:44:42.755744 20492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/ts-1/data/instance:
uuid: "cb66e0b3eac24a2f89d26b204ad4858a"
format_stamp: "Formatted at 2026-08-06 22:44:42 on dist-test-slave-h3ft"
server_key: "fbfd2918d99f6df16df2f58d92c71a55"
server_key_iv: "36068d9879293040e7c55d7c5070d7f7"
server_key_version: "encryptionkey@0"
I20260806 22:44:42.756196 20492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/ts-1/wal/instance:
uuid: "cb66e0b3eac24a2f89d26b204ad4858a"
format_stamp: "Formatted at 2026-08-06 22:44:42 on dist-test-slave-h3ft"
server_key: "fbfd2918d99f6df16df2f58d92c71a55"
server_key_iv: "36068d9879293040e7c55d7c5070d7f7"
server_key_version: "encryptionkey@0"
I20260806 22:44:42.757850 20492 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 22:44:42.758926 20515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:42.759195 20492 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:44:42.759354 20492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/ts-1/wal
uuid: "cb66e0b3eac24a2f89d26b204ad4858a"
format_stamp: "Formatted at 2026-08-06 22:44:42 on dist-test-slave-h3ft"
server_key: "fbfd2918d99f6df16df2f58d92c71a55"
server_key_iv: "36068d9879293040e7c55d7c5070d7f7"
server_key_version: "encryptionkey@0"
I20260806 22:44:42.759460 20492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:42.773717 20492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:42.776600 20492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:42.776798 20492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:42.777547 20492 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:44:42.777617 20492 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:42.777664 20492 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:44:42.777709 20492 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 22:44:42 dist-test-slave-h3ft krb5kdc[20125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056282, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.194@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:44:42.787047 20492 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.194:42677
I20260806 22:44:42.787170 20688 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.194:42677 every 8 connection(s)
I20260806 22:44:42.787503 20492 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
I20260806 22:44:42.791190 20262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:42.778207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:43805 (local address 127.18.45.254:42241)
0806 22:44:42.778305 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:42.778308 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:42.778612 (+ 304us) server_negotiation.cc:407] Connection header received
0806 22:44:42.779571 (+ 959us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:42.779573 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:42.779596 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:42.779664 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:42.780853 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:42.781880 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:42.783946 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:42.784215 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:42.784326 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:42.787728 (+ 3402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:42.787732 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:42.787733 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:42.787743 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:42.789453 (+ 1710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:42.789972 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:42.789974 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:42.789974 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:42.790057 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:42.790420 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:42.790421 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:42.790421 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:42.790584 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:42.790652 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:42.790950 (+ 298us) server_negotiation.cc:299] Negotiation successful
0806 22:44:42.791100 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 22:44:42.791421 20528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:42.778173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:42241 (local address 127.18.45.194:43805)
0806 22:44:42.778417 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:42.778492 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:42.779401 (+ 909us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:42.779796 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:42.779803 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:42.780297 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:42.780719 (+ 422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:42.780728 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:42.782012 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:42.782013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:42.783828 (+ 1815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:42.783833 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:42.784352 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:42.784353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:42.784548 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:42.785342 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:44:42.785355 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:42.787566 (+ 2211us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:42.789541 (+ 1975us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:42.789549 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:42.789551 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:42.789814 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:42.790145 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:42.790146 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:42.790146 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:42.790275 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:42.790654 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:42.790660 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:42.790812 (+ 152us) client_negotiation.cc:769] Sending connection context
0806 22:44:42.790902 (+ 90us) client_negotiation.cc:240] Negotiation successful
0806 22:44:42.791144 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":119,"threads_started":1}
I20260806 22:44:42.792033 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 20492
I20260806 22:44:42.792044 20689 heartbeater.cc:347] Connected to a master server at 127.18.45.254:42241
I20260806 22:44:42.792151 20689 heartbeater.cc:464] Registering TS with master...
I20260806 22:44:42.792155 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/ts-1/wal/instance
I20260806 22:44:42.792388 18615 external_mini_cluster.cc:1428] Setting key d1d70332f3b547db47d8dfa7b8ed307f
I20260806 22:44:42.792392 20689 heartbeater.cc:511] Master 127.18.45.254:42241 requested a full tablet report, sending...
I20260806 22:44:42.792949 20189 ts_manager.cc:194] Registered new tserver with Master: cb66e0b3eac24a2f89d26b204ad4858a (127.18.45.194:42677)
I20260806 22:44:42.793516 20189 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.45.194@KRBTEST.COM'} at 127.18.45.194:43805
I20260806 22:44:42.800930 20697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:42.793435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:42241 (local address 127.18.45.194:54909)
0806 22:44:42.793610 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:42.793624 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:42.793692 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:42.793832 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:42.793833 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:42.794109 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:42.794214 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:42.794216 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:42.794963 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:42.794964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:42.795966 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:42.795969 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:42.796436 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:42.796437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:42.796553 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:42.797100 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:44:42.797106 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:42.797569 (+ 463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:42.799610 (+ 2041us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:42.799611 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:42.799612 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:42.799782 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:42.800050 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:42.800051 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:42.800052 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:42.800149 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:42.800612 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:42.800613 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:42.800664 (+ 51us) client_negotiation.cc:769] Sending connection context
0806 22:44:42.800736 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 22:44:42.800837 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":89,"threads_started":1}
I20260806 22:44:42.801033 20262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:42.793648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:54909 (local address 127.18.45.254:42241)
0806 22:44:42.793688 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:42.793690 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:42.793700 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:44:42.793762 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:42.793763 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:42.793774 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:42.793828 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:42.794310 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:42.794855 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:42.796041 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:42.796357 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:42.796435 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:42.797651 (+ 1216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:42.797653 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:42.797654 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:42.797661 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:42.799524 (+ 1863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:42.799858 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:42.799860 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:42.799861 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:42.799930 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:42.800239 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:42.800240 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:42.800241 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:42.800507 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:42.800593 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:42.800796 (+ 203us) server_negotiation.cc:299] Negotiation successful
0806 22:44:42.800897 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
WARNING: no policy specified for kudu/127.18.45.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab.
Entry for principal kudu/127.18.45.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab.
I20260806 22:44:42.828054 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.195:0
--local_ip_for_outbound_sockets=127.18.45.195
--webserver_interface=127.18.45.195
--webserver_port=0
--tserver_master_addrs=127.18.45.254:42241
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab
--principal=kudu/127.18.45.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:40553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:44:42.931682 20700 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:42.931957 20700 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:42.931986 20700 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:42.933291 20700 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:42.933339 20700 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:42.933360 20700 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:44:42.933379 20700 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.195
I20260806 22:44:42.934983 20700 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:40553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab
--principal=kudu/127.18.45.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.45.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.45.254:42241
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20700
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.195
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:44:42.935339 20700 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:42.935624 20700 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:42.941893 20706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:42.941874 20709 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:42.941881 20707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:42.942231 20700 server_base.cc:1034] running on GCE node
I20260806 22:44:42.942543 20700 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:42.942839 20700 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:42.944038 20700 hybrid_clock.cc:648] HybridClock initialized: now 1786056282944013 us; error 45 us; skew 500 ppm
Aug 06 22:44:42 dist-test-slave-h3ft krb5kdc[20125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056282, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:42.947535 20700 init.cc:377] Logged in from keytab as kudu/127.18.45.195@KRBTEST.COM (short username kudu)
I20260806 22:44:42.948155 20700 webserver.cc:460] Webserver started at http://127.18.45.195:41695/ using document root <none> and password file <none>
I20260806 22:44:42.948467 20700 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:42.948551 20700 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:42.948742 20700 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:44:42.950176 20700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/ts-2/data/instance:
uuid: "dd6c97f5a88b46478293f12b101f8c7c"
format_stamp: "Formatted at 2026-08-06 22:44:42 on dist-test-slave-h3ft"
server_key: "7427c851a2283c4099f6f8c414734347"
server_key_iv: "a68f77ed5427d6a6d8af5bab876cf4d8"
server_key_version: "encryptionkey@0"
I20260806 22:44:42.950907 20700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/ts-2/wal/instance:
uuid: "dd6c97f5a88b46478293f12b101f8c7c"
format_stamp: "Formatted at 2026-08-06 22:44:42 on dist-test-slave-h3ft"
server_key: "7427c851a2283c4099f6f8c414734347"
server_key_iv: "a68f77ed5427d6a6d8af5bab876cf4d8"
server_key_version: "encryptionkey@0"
I20260806 22:44:42.952988 20700 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260806 22:44:42.954123 20718 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:42.954411 20700 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:44:42.954546 20700 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/ts-2/wal
uuid: "dd6c97f5a88b46478293f12b101f8c7c"
format_stamp: "Formatted at 2026-08-06 22:44:42 on dist-test-slave-h3ft"
server_key: "7427c851a2283c4099f6f8c414734347"
server_key_iv: "a68f77ed5427d6a6d8af5bab876cf4d8"
server_key_version: "encryptionkey@0"
I20260806 22:44:42.954650 20700 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:42.975936 20700 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:42.978812 20700 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:42.979041 20700 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:42.979732 20700 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:44:42.979802 20700 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:42.979848 20700 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:44:42.979866 20700 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:42.988575 20700 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.195:38227
I20260806 22:44:42.988734 20869 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.195:38227 every 8 connection(s)
I20260806 22:44:42.989071 20700 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
Aug 06 22:44:42 dist-test-slave-h3ft krb5kdc[20125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056282, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.195@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:44:42.993202 20262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:42.980445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.195:57853 (local address 127.18.45.254:42241)
0806 22:44:42.980524 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:42.980527 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:42.980803 (+ 276us) server_negotiation.cc:407] Connection header received
0806 22:44:42.981557 (+ 754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:42.981559 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:42.981579 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:42.981663 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:42.982789 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:42.983830 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:42.985877 (+ 2047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:42.986292 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:42.986383 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:42.989801 (+ 3418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:42.989803 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:42.989804 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:42.989814 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:42.991479 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:42.991960 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:42.991962 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:42.991963 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:42.992045 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:42.992364 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:42.992365 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:42.992365 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:42.992531 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:42.992629 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:42.992966 (+ 337us) server_negotiation.cc:299] Negotiation successful
0806 22:44:42.993077 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 22:44:42.993468 20731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:42.980430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:42241 (local address 127.18.45.195:57853)
0806 22:44:42.980656 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:42.980698 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:42.981374 (+ 676us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:42.981813 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:42.981819 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:42.982276 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:42.982655 (+ 379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:42.982663 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:42.983980 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:42.983982 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:42.985771 (+ 1789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:42.985777 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:42.986402 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:42.986403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:42.986577 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:42.987349 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:44:42.987363 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:42.989656 (+ 2293us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:42.991588 (+ 1932us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:42.991597 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:42.991599 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:42.991867 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:42.992135 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:42.992136 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:42.992136 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:42.992270 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:42.992616 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:42.992625 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:42.992803 (+ 178us) client_negotiation.cc:769] Sending connection context
0806 22:44:42.992943 (+ 140us) client_negotiation.cc:240] Negotiation successful
0806 22:44:42.993160 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":127,"threads_started":1}
I20260806 22:44:42.994146 20870 heartbeater.cc:347] Connected to a master server at 127.18.45.254:42241
I20260806 22:44:42.994249 20870 heartbeater.cc:464] Registering TS with master...
I20260806 22:44:42.994491 20870 heartbeater.cc:511] Master 127.18.45.254:42241 requested a full tablet report, sending...
I20260806 22:44:42.995049 20187 ts_manager.cc:194] Registered new tserver with Master: dd6c97f5a88b46478293f12b101f8c7c (127.18.45.195:38227)
I20260806 22:44:42.995623 20187 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.45.195@KRBTEST.COM'} at 127.18.45.195:57853
I20260806 22:44:42.995690 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 20700
I20260806 22:44:42.995806 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/ts-2/wal/instance
I20260806 22:44:42.996038 18615 external_mini_cluster.cc:1428] Setting key 5e0de27b8802166ab3dcd2ee3e59696d
I20260806 22:44:42.997146 18615 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:44:43.003949 20262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:42.995681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.195:53341 (local address 127.18.45.254:42241)
0806 22:44:42.995793 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:42.995796 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:42.996978 (+ 1182us) server_negotiation.cc:407] Connection header received
0806 22:44:42.997015 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:42.997015 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:42.997025 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:42.997079 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:42.997595 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:42.998329 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:42.999178 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:42.999483 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:42.999560 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:43.000881 (+ 1321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:43.000883 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:43.000883 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:43.000890 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:43.002468 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:43.002914 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:43.002916 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:43.002916 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:43.002982 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:43.003315 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:43.003316 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:43.003317 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:43.003447 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:43.003611 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:43.003792 (+ 181us) server_negotiation.cc:299] Negotiation successful
0806 22:44:43.003868 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260806 22:44:43.004050 20880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:42.995731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:42241 (local address 127.18.45.195:53341)
0806 22:44:42.996008 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:42.996023 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:42.996100 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:42.997112 (+ 1012us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:42.997114 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:42.997380 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:42.997506 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:42.997509 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:42.998464 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:42.998466 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:42.999081 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:42.999083 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:42.999587 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:42.999589 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:42.999660 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:43.000249 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:44:43.000256 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:43.000781 (+ 525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:43.002563 (+ 1782us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:43.002564 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:43.002564 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:43.002814 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:43.003099 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:43.003101 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:43.003101 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:43.003193 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:43.003586 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:43.003587 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:43.003671 (+ 84us) client_negotiation.cc:769] Sending connection context
0806 22:44:43.003756 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 22:44:43.003899 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":99,"threads_started":1}
Aug 06 22:44:43 dist-test-slave-h3ft krb5kdc[20125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056283, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 22:44:43 dist-test-slave-h3ft krb5kdc[20125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056283, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:44:43.027449 20262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:43.018946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37648 (local address 127.18.45.254:42241)
0806 22:44:43.019045 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:43.019048 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:43.019063 (+ 15us) server_negotiation.cc:407] Connection header received
0806 22:44:43.019147 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:43.019148 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:43.019164 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:43.019230 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:43.019895 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.020474 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.021288 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.021504 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.021597 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:43.024506 (+ 2909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:43.024508 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:43.024509 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:43.024518 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:43.026160 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:43.026546 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:43.026547 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:43.026547 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:43.026606 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:43.026828 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:43.026828 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:43.026829 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:43.026986 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:43.027144 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:43.027281 (+ 137us) server_negotiation.cc:299] Negotiation successful
0806 22:44:43.027358 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 22:44:43.029565 20187 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37648:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:44:43.031837 20187 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:44:43.039460 20896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:43.034901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.194:42677 (local address 127.0.0.1:36320)
0806 22:44:43.035172 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:43.035221 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:43.035304 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:43.035645 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:43.035649 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:43.035651 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:44:43.035750 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:43.035752 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.036816 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.036816 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:43.038112 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:43.038115 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.039094 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.039095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:43.039187 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:43.039204 (+ 17us) client_negotiation.cc:769] Sending connection context
0806 22:44:43.039284 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 22:44:43.039351 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":78,"threads_started":1}
I20260806 22:44:43.039475 20900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:43.035139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36320 (local address 127.18.45.194:42677)
0806 22:44:43.035353 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:43.035360 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:43.035384 (+ 24us) server_negotiation.cc:407] Connection header received
0806 22:44:43.035440 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:43.035442 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:43.035464 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:43.035646 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:44:43.035842 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.036723 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.038220 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.038964 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.039093 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:43.039180 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:43.039302 (+ 122us) server_negotiation.cc:299] Negotiation successful
0806 22:44:43.039354 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":109,"threads_started":1}
I20260806 22:44:43.039913 20903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:43.035518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46242 (local address 127.18.45.193:33651)
0806 22:44:43.035845 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:43.035849 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:43.035860 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:44:43.035909 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:43.035911 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:43.035926 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:43.036012 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:44:43.036126 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.037257 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.038584 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.039338 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.039432 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:43.039497 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:43.039738 (+ 241us) server_negotiation.cc:299] Negotiation successful
0806 22:44:43.039792 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":105,"threads_started":1}
I20260806 22:44:43.039685 20897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:43.034932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.195:38227 (local address 127.0.0.1:58096)
0806 22:44:43.035387 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:43.035400 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:43.035463 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:43.035920 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:43.035921 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:43.035923 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:44:43.035988 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:43.035990 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.037395 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.037395 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:43.038366 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:43.038369 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.039450 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.039451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:43.039518 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:43.039522 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:44:43.039556 (+ 34us) client_negotiation.cc:240] Negotiation successful
0806 22:44:43.039600 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":150,"threads_started":1}
I20260806 22:44:43.040139 20899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:43.035055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58096 (local address 127.18.45.195:38227)
0806 22:44:43.035704 (+ 649us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:43.035709 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:43.035722 (+ 13us) server_negotiation.cc:407] Connection header received
0806 22:44:43.035771 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:43.035773 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:43.035792 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:43.036070 (+ 278us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:44:43.036147 (+ 77us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.037280 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.038469 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.039344 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.039795 (+ 451us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:43.039934 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:43.039961 (+ 27us) server_negotiation.cc:299] Negotiation successful
0806 22:44:43.040014 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":526,"thread_start_us":109,"threads_started":1}
I20260806 22:44:43.039816 20902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:43.035211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.193:33651 (local address 127.0.0.1:46242)
0806 22:44:43.035451 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:43.035460 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:43.035510 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:43.036008 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:43.036009 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:43.036012 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:44:43.036055 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:43.036056 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.037356 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.037356 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:43.038488 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:43.038490 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.039528 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.039529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:43.039585 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:43.039588 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:44:43.039634 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 22:44:43.039668 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"mutex_wait_us":33,"thread_start_us":76,"threads_started":1}
I20260806 22:44:43.041121 20787 tablet_service.cc:1467] Processing CreateTablet for tablet efaf9a4f788447cc81f57f0f98d91e3d (DEFAULT_TABLE table=test-table [id=ab83a5ca350b496eb894b95a8d59dda0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:44:43.041529 20787 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet efaf9a4f788447cc81f57f0f98d91e3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:43.043545 20599 tablet_service.cc:1467] Processing CreateTablet for tablet efaf9a4f788447cc81f57f0f98d91e3d (DEFAULT_TABLE table=test-table [id=ab83a5ca350b496eb894b95a8d59dda0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:44:43.044000 20599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet efaf9a4f788447cc81f57f0f98d91e3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:43.045203 20904 tablet_bootstrap.cc:492] T efaf9a4f788447cc81f57f0f98d91e3d P dd6c97f5a88b46478293f12b101f8c7c: Bootstrap starting.
I20260806 22:44:43.046514 20904 tablet_bootstrap.cc:654] T efaf9a4f788447cc81f57f0f98d91e3d P dd6c97f5a88b46478293f12b101f8c7c: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:43.046964 20904 log.cc:824] T efaf9a4f788447cc81f57f0f98d91e3d P dd6c97f5a88b46478293f12b101f8c7c: Log is configured to *not* fsync() on all Append() calls
I20260806 22:44:43.047611 20386 tablet_service.cc:1467] Processing CreateTablet for tablet efaf9a4f788447cc81f57f0f98d91e3d (DEFAULT_TABLE table=test-table [id=ab83a5ca350b496eb894b95a8d59dda0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:44:43.047958 20386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet efaf9a4f788447cc81f57f0f98d91e3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:43.048043 20904 tablet_bootstrap.cc:492] T efaf9a4f788447cc81f57f0f98d91e3d P dd6c97f5a88b46478293f12b101f8c7c: No bootstrap required, opened a new log
I20260806 22:44:43.048117 20904 ts_tablet_manager.cc:1397] T efaf9a4f788447cc81f57f0f98d91e3d P dd6c97f5a88b46478293f12b101f8c7c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:44:43.048511 20907 tablet_bootstrap.cc:492] T efaf9a4f788447cc81f57f0f98d91e3d P cb66e0b3eac24a2f89d26b204ad4858a: Bootstrap starting.
I20260806 22:44:43.049261 20907 tablet_bootstrap.cc:654] T efaf9a4f788447cc81f57f0f98d91e3d P cb66e0b3eac24a2f89d26b204ad4858a: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:43.049692 20907 log.cc:824] T efaf9a4f788447cc81f57f0f98d91e3d P cb66e0b3eac24a2f89d26b204ad4858a: Log is configured to *not* fsync() on all Append() calls
I20260806 22:44:43.050211 20904 raft_consensus.cc:348] T efaf9a4f788447cc81f57f0f98d91e3d P dd6c97f5a88b46478293f12b101f8c7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd6c97f5a88b46478293f12b101f8c7c" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 38227 } } peers { permanent_uuid: "68a1a25643ff4b9aad520e2d7a1f5008" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33651 } } peers { permanent_uuid: "cb66e0b3eac24a2f89d26b204ad4858a" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 42677 } }
I20260806 22:44:43.050416 20904 raft_consensus.cc:374] T efaf9a4f788447cc81f57f0f98d91e3d P dd6c97f5a88b46478293f12b101f8c7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:43.050465 20904 raft_consensus.cc:729] T efaf9a4f788447cc81f57f0f98d91e3d P dd6c97f5a88b46478293f12b101f8c7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd6c97f5a88b46478293f12b101f8c7c, State: Initialized, Role: FOLLOWER
I20260806 22:44:43.050629 20904 consensus_queue.cc:260] T efaf9a4f788447cc81f57f0f98d91e3d P dd6c97f5a88b46478293f12b101f8c7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd6c97f5a88b46478293f12b101f8c7c" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 38227 } } peers { permanent_uuid: "68a1a25643ff4b9aad520e2d7a1f5008" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33651 } } peers { permanent_uuid: "cb66e0b3eac24a2f89d26b204ad4858a" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 42677 } }
I20260806 22:44:43.050969 20904 ts_tablet_manager.cc:1428] T efaf9a4f788447cc81f57f0f98d91e3d P dd6c97f5a88b46478293f12b101f8c7c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:44:43.051141 20870 heartbeater.cc:503] Master 127.18.45.254:42241 was elected leader, sending a full tablet report...
I20260806 22:44:43.051126 20907 tablet_bootstrap.cc:492] T efaf9a4f788447cc81f57f0f98d91e3d P cb66e0b3eac24a2f89d26b204ad4858a: No bootstrap required, opened a new log
I20260806 22:44:43.051234 20907 ts_tablet_manager.cc:1397] T efaf9a4f788447cc81f57f0f98d91e3d P cb66e0b3eac24a2f89d26b204ad4858a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:44:43.052363 20911 tablet_bootstrap.cc:492] T efaf9a4f788447cc81f57f0f98d91e3d P 68a1a25643ff4b9aad520e2d7a1f5008: Bootstrap starting.
I20260806 22:44:43.053027 20911 tablet_bootstrap.cc:654] T efaf9a4f788447cc81f57f0f98d91e3d P 68a1a25643ff4b9aad520e2d7a1f5008: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:43.053413 20911 log.cc:824] T efaf9a4f788447cc81f57f0f98d91e3d P 68a1a25643ff4b9aad520e2d7a1f5008: Log is configured to *not* fsync() on all Append() calls
I20260806 22:44:43.053352 20907 raft_consensus.cc:348] T efaf9a4f788447cc81f57f0f98d91e3d P cb66e0b3eac24a2f89d26b204ad4858a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd6c97f5a88b46478293f12b101f8c7c" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 38227 } } peers { permanent_uuid: "68a1a25643ff4b9aad520e2d7a1f5008" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33651 } } peers { permanent_uuid: "cb66e0b3eac24a2f89d26b204ad4858a" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 42677 } }
I20260806 22:44:43.053485 20907 raft_consensus.cc:374] T efaf9a4f788447cc81f57f0f98d91e3d P cb66e0b3eac24a2f89d26b204ad4858a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:43.053547 20907 raft_consensus.cc:729] T efaf9a4f788447cc81f57f0f98d91e3d P cb66e0b3eac24a2f89d26b204ad4858a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb66e0b3eac24a2f89d26b204ad4858a, State: Initialized, Role: FOLLOWER
I20260806 22:44:43.053722 20907 consensus_queue.cc:260] T efaf9a4f788447cc81f57f0f98d91e3d P cb66e0b3eac24a2f89d26b204ad4858a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd6c97f5a88b46478293f12b101f8c7c" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 38227 } } peers { permanent_uuid: "68a1a25643ff4b9aad520e2d7a1f5008" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33651 } } peers { permanent_uuid: "cb66e0b3eac24a2f89d26b204ad4858a" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 42677 } }
I20260806 22:44:43.054098 20907 ts_tablet_manager.cc:1428] T efaf9a4f788447cc81f57f0f98d91e3d P cb66e0b3eac24a2f89d26b204ad4858a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:44:43.054124 20689 heartbeater.cc:503] Master 127.18.45.254:42241 was elected leader, sending a full tablet report...
I20260806 22:44:43.054291 20911 tablet_bootstrap.cc:492] T efaf9a4f788447cc81f57f0f98d91e3d P 68a1a25643ff4b9aad520e2d7a1f5008: No bootstrap required, opened a new log
I20260806 22:44:43.054350 20911 ts_tablet_manager.cc:1397] T efaf9a4f788447cc81f57f0f98d91e3d P 68a1a25643ff4b9aad520e2d7a1f5008: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:44:43.056380 20911 raft_consensus.cc:348] T efaf9a4f788447cc81f57f0f98d91e3d P 68a1a25643ff4b9aad520e2d7a1f5008 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd6c97f5a88b46478293f12b101f8c7c" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 38227 } } peers { permanent_uuid: "68a1a25643ff4b9aad520e2d7a1f5008" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33651 } } peers { permanent_uuid: "cb66e0b3eac24a2f89d26b204ad4858a" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 42677 } }
I20260806 22:44:43.056565 20911 raft_consensus.cc:374] T efaf9a4f788447cc81f57f0f98d91e3d P 68a1a25643ff4b9aad520e2d7a1f5008 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:43.056591 20911 raft_consensus.cc:729] T efaf9a4f788447cc81f57f0f98d91e3d P 68a1a25643ff4b9aad520e2d7a1f5008 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68a1a25643ff4b9aad520e2d7a1f5008, State: Initialized, Role: FOLLOWER
I20260806 22:44:43.056681 20911 consensus_queue.cc:260] T efaf9a4f788447cc81f57f0f98d91e3d P 68a1a25643ff4b9aad520e2d7a1f5008 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd6c97f5a88b46478293f12b101f8c7c" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 38227 } } peers { permanent_uuid: "68a1a25643ff4b9aad520e2d7a1f5008" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33651 } } peers { permanent_uuid: "cb66e0b3eac24a2f89d26b204ad4858a" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 42677 } }
I20260806 22:44:43.056921 20911 ts_tablet_manager.cc:1428] T efaf9a4f788447cc81f57f0f98d91e3d P 68a1a25643ff4b9aad520e2d7a1f5008: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:44:43.057008 20481 heartbeater.cc:503] Master 127.18.45.254:42241 was elected leader, sending a full tablet report...
I20260806 22:44:43.075378 20913 raft_consensus.cc:482] T efaf9a4f788447cc81f57f0f98d91e3d P cb66e0b3eac24a2f89d26b204ad4858a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:44:43.075588 20913 raft_consensus.cc:504] T efaf9a4f788447cc81f57f0f98d91e3d P cb66e0b3eac24a2f89d26b204ad4858a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd6c97f5a88b46478293f12b101f8c7c" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 38227 } } peers { permanent_uuid: "68a1a25643ff4b9aad520e2d7a1f5008" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33651 } } peers { permanent_uuid: "cb66e0b3eac24a2f89d26b204ad4858a" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 42677 } }
I20260806 22:44:43.076128 20913 leader_election.cc:288] T efaf9a4f788447cc81f57f0f98d91e3d P cb66e0b3eac24a2f89d26b204ad4858a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dd6c97f5a88b46478293f12b101f8c7c (127.18.45.195:38227), 68a1a25643ff4b9aad520e2d7a1f5008 (127.18.45.193:33651)
I20260806 22:44:43.080780 20528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:43.076489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.195:38227 (local address 127.18.45.194:39903)
0806 22:44:43.076566 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:43.076586 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:43.076654 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:43.076839 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:43.076841 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:43.076844 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:44:43.076944 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:43.076993 (+ 49us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.078045 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.078046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:43.079527 (+ 1481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:43.079531 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.080391 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.080393 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:43.080529 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:43.080535 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 22:44:43.080600 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 22:44:43.080661 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29,"mutex_wait_us":116}
I20260806 22:44:43.080796 20916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:43.076621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.193:33651 (local address 127.18.45.194:36481)
0806 22:44:43.076852 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:43.076865 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:43.076937 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:43.077076 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:43.077076 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:43.077078 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:44:43.077165 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:43.077166 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.078045 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.078046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:43.079528 (+ 1482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:43.079531 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.080392 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.080393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:43.080529 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:43.080535 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 22:44:43.080600 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 22:44:43.080661 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"mutex_wait_us":65,"thread_start_us":149,"threads_started":1}
I20260806 22:44:43.080816 20899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:43.076618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:39903 (local address 127.18.45.195:38227)
0806 22:44:43.076683 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:43.076686 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:43.076709 (+ 23us) server_negotiation.cc:407] Connection header received
0806 22:44:43.076745 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:43.076746 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:43.076757 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:43.076810 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:44:43.077073 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.077925 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.079664 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.080287 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.080378 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:43.080392 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:43.080643 (+ 251us) server_negotiation.cc:299] Negotiation successful
0806 22:44:43.080711 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 22:44:43.081213 20903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:43.076744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:36481 (local address 127.18.45.193:33651)
0806 22:44:43.076809 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:43.076812 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:43.076924 (+ 112us) server_negotiation.cc:407] Connection header received
0806 22:44:43.076993 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:43.076994 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:43.077002 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:43.077073 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:44:43.077239 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.077950 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.079660 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.080279 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.080380 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:43.080392 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:43.081044 (+ 652us) server_negotiation.cc:299] Negotiation successful
0806 22:44:43.081120 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 22:44:43.081532 20416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efaf9a4f788447cc81f57f0f98d91e3d" candidate_uuid: "cb66e0b3eac24a2f89d26b204ad4858a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68a1a25643ff4b9aad520e2d7a1f5008" is_pre_election: true
I20260806 22:44:43.081532 20812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efaf9a4f788447cc81f57f0f98d91e3d" candidate_uuid: "cb66e0b3eac24a2f89d26b204ad4858a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd6c97f5a88b46478293f12b101f8c7c" is_pre_election: true
I20260806 22:44:43.081745 20812 raft_consensus.cc:2454] T efaf9a4f788447cc81f57f0f98d91e3d P dd6c97f5a88b46478293f12b101f8c7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb66e0b3eac24a2f89d26b204ad4858a in term 0.
I20260806 22:44:43.081745 20416 raft_consensus.cc:2454] T efaf9a4f788447cc81f57f0f98d91e3d P 68a1a25643ff4b9aad520e2d7a1f5008 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb66e0b3eac24a2f89d26b204ad4858a in term 0.
I20260806 22:44:43.082044 20516 leader_election.cc:302] T efaf9a4f788447cc81f57f0f98d91e3d P cb66e0b3eac24a2f89d26b204ad4858a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68a1a25643ff4b9aad520e2d7a1f5008, cb66e0b3eac24a2f89d26b204ad4858a; no voters:
I20260806 22:44:43.082206 20913 raft_consensus.cc:2781] T efaf9a4f788447cc81f57f0f98d91e3d P cb66e0b3eac24a2f89d26b204ad4858a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:44:43.082273 20913 raft_consensus.cc:482] T efaf9a4f788447cc81f57f0f98d91e3d P cb66e0b3eac24a2f89d26b204ad4858a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:44:43.082291 20913 raft_consensus.cc:3037] T efaf9a4f788447cc81f57f0f98d91e3d P cb66e0b3eac24a2f89d26b204ad4858a [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:43.083587 20913 raft_consensus.cc:504] T efaf9a4f788447cc81f57f0f98d91e3d P cb66e0b3eac24a2f89d26b204ad4858a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd6c97f5a88b46478293f12b101f8c7c" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 38227 } } peers { permanent_uuid: "68a1a25643ff4b9aad520e2d7a1f5008" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33651 } } peers { permanent_uuid: "cb66e0b3eac24a2f89d26b204ad4858a" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 42677 } }
I20260806 22:44:43.083731 20913 leader_election.cc:288] T efaf9a4f788447cc81f57f0f98d91e3d P cb66e0b3eac24a2f89d26b204ad4858a [CANDIDATE]: Term 1 election: Requested vote from peers dd6c97f5a88b46478293f12b101f8c7c (127.18.45.195:38227), 68a1a25643ff4b9aad520e2d7a1f5008 (127.18.45.193:33651)
I20260806 22:44:43.084028 20812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efaf9a4f788447cc81f57f0f98d91e3d" candidate_uuid: "cb66e0b3eac24a2f89d26b204ad4858a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd6c97f5a88b46478293f12b101f8c7c"
I20260806 22:44:43.084098 20416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efaf9a4f788447cc81f57f0f98d91e3d" candidate_uuid: "cb66e0b3eac24a2f89d26b204ad4858a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68a1a25643ff4b9aad520e2d7a1f5008"
I20260806 22:44:43.084164 20812 raft_consensus.cc:3037] T efaf9a4f788447cc81f57f0f98d91e3d P dd6c97f5a88b46478293f12b101f8c7c [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:43.084199 20416 raft_consensus.cc:3037] T efaf9a4f788447cc81f57f0f98d91e3d P 68a1a25643ff4b9aad520e2d7a1f5008 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:43.085376 20812 raft_consensus.cc:2454] T efaf9a4f788447cc81f57f0f98d91e3d P dd6c97f5a88b46478293f12b101f8c7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb66e0b3eac24a2f89d26b204ad4858a in term 1.
I20260806 22:44:43.085441 20416 raft_consensus.cc:2454] T efaf9a4f788447cc81f57f0f98d91e3d P 68a1a25643ff4b9aad520e2d7a1f5008 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb66e0b3eac24a2f89d26b204ad4858a in term 1.
I20260806 22:44:43.085670 20516 leader_election.cc:302] T efaf9a4f788447cc81f57f0f98d91e3d P cb66e0b3eac24a2f89d26b204ad4858a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb66e0b3eac24a2f89d26b204ad4858a, dd6c97f5a88b46478293f12b101f8c7c; no voters:
I20260806 22:44:43.085790 20913 raft_consensus.cc:2781] T efaf9a4f788447cc81f57f0f98d91e3d P cb66e0b3eac24a2f89d26b204ad4858a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:44:43.085992 20913 raft_consensus.cc:686] T efaf9a4f788447cc81f57f0f98d91e3d P cb66e0b3eac24a2f89d26b204ad4858a [term 1 LEADER]: Becoming Leader. State: Replica: cb66e0b3eac24a2f89d26b204ad4858a, State: Running, Role: LEADER
I20260806 22:44:43.086117 20913 consensus_queue.cc:237] T efaf9a4f788447cc81f57f0f98d91e3d P cb66e0b3eac24a2f89d26b204ad4858a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd6c97f5a88b46478293f12b101f8c7c" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 38227 } } peers { permanent_uuid: "68a1a25643ff4b9aad520e2d7a1f5008" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33651 } } peers { permanent_uuid: "cb66e0b3eac24a2f89d26b204ad4858a" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 42677 } }
I20260806 22:44:43.087293 20187 catalog_manager.cc:5495] T efaf9a4f788447cc81f57f0f98d91e3d P cb66e0b3eac24a2f89d26b204ad4858a reported cstate change: term changed from 0 to 1, leader changed from <none> to cb66e0b3eac24a2f89d26b204ad4858a (127.18.45.194). New cstate: current_term: 1 leader_uuid: "cb66e0b3eac24a2f89d26b204ad4858a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd6c97f5a88b46478293f12b101f8c7c" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 38227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68a1a25643ff4b9aad520e2d7a1f5008" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb66e0b3eac24a2f89d26b204ad4858a" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 42677 } health_report { overall_health: HEALTHY } } }
W20260806 22:44:43.095232 20482 tablet.cc:2369] T efaf9a4f788447cc81f57f0f98d91e3d P 68a1a25643ff4b9aad520e2d7a1f5008: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:44:43.101747 20900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:43.098548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36334 (local address 127.18.45.194:42677)
0806 22:44:43.098620 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:43.098623 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:43.098662 (+ 39us) server_negotiation.cc:407] Connection header received
0806 22:44:43.098712 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:43.098713 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:43.098737 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:43.098789 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:44:43.099042 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.099606 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.100555 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.100770 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.100840 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:43.101190 (+ 350us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:44:43.101350 (+ 160us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:44:43.101423 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:43.101530 (+ 107us) server_negotiation.cc:299] Negotiation successful
0806 22:44:43.101561 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 22:44:43.104543 20812 raft_consensus.cc:1261] T efaf9a4f788447cc81f57f0f98d91e3d P dd6c97f5a88b46478293f12b101f8c7c [term 1 FOLLOWER]: Refusing update from remote peer cb66e0b3eac24a2f89d26b204ad4858a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:44:43.104543 20416 raft_consensus.cc:1261] T efaf9a4f788447cc81f57f0f98d91e3d P 68a1a25643ff4b9aad520e2d7a1f5008 [term 1 FOLLOWER]: Refusing update from remote peer cb66e0b3eac24a2f89d26b204ad4858a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:44:43.105147 20913 consensus_queue.cc:1035] T efaf9a4f788447cc81f57f0f98d91e3d P cb66e0b3eac24a2f89d26b204ad4858a [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd6c97f5a88b46478293f12b101f8c7c" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 38227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:44:43.107730 20913 consensus_queue.cc:1035] T efaf9a4f788447cc81f57f0f98d91e3d P cb66e0b3eac24a2f89d26b204ad4858a [LEADER]: Connected to new peer: Peer: permanent_uuid: "68a1a25643ff4b9aad520e2d7a1f5008" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:44:43.108798 20925 mvcc.cc:205] Tried to move back new op lower bound from 7315686535591604224 to 7315686535521689600. Current Snapshot: MvccSnapshot[applied={T|T < 7315686535591604224}]
I20260806 22:44:43.114050 20187 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37648:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:44:43.114243 20187 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37648:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:44:43.115442 20187 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ece890efc8674a71b7ee1a70429fc1bb: Sending DeleteTablet for 3 replicas of tablet efaf9a4f788447cc81f57f0f98d91e3d
I20260806 22:44:43.116139 20787 tablet_service.cc:1514] Processing DeleteTablet for tablet efaf9a4f788447cc81f57f0f98d91e3d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:44:43 UTC) from {username='kudu', principal='kudu/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:58096
I20260806 22:44:43.116231 20599 tablet_service.cc:1514] Processing DeleteTablet for tablet efaf9a4f788447cc81f57f0f98d91e3d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:44:43 UTC) from {username='kudu', principal='kudu/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:36320
I20260806 22:44:43.116495 20386 tablet_service.cc:1514] Processing DeleteTablet for tablet efaf9a4f788447cc81f57f0f98d91e3d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:44:43 UTC) from {username='kudu', principal='kudu/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:46242
I20260806 22:44:43.116654 20943 tablet_replica.cc:330] T efaf9a4f788447cc81f57f0f98d91e3d P cb66e0b3eac24a2f89d26b204ad4858a: stopping tablet replica
I20260806 22:44:43.116823 20942 tablet_replica.cc:330] T efaf9a4f788447cc81f57f0f98d91e3d P dd6c97f5a88b46478293f12b101f8c7c: stopping tablet replica
I20260806 22:44:43.116834 20943 raft_consensus.cc:2229] T efaf9a4f788447cc81f57f0f98d91e3d P cb66e0b3eac24a2f89d26b204ad4858a [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:44:43.116995 20942 raft_consensus.cc:2229] T efaf9a4f788447cc81f57f0f98d91e3d P dd6c97f5a88b46478293f12b101f8c7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:44:43.116882 20187 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37648:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:44:43.117107 20187 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:44:43.117169 20942 raft_consensus.cc:2258] T efaf9a4f788447cc81f57f0f98d91e3d P dd6c97f5a88b46478293f12b101f8c7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:44:43.117352 20944 tablet_replica.cc:330] T efaf9a4f788447cc81f57f0f98d91e3d P 68a1a25643ff4b9aad520e2d7a1f5008: stopping tablet replica
I20260806 22:44:43.117532 20944 raft_consensus.cc:2229] T efaf9a4f788447cc81f57f0f98d91e3d P 68a1a25643ff4b9aad520e2d7a1f5008 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:44:43.117619 20943 raft_consensus.cc:2258] T efaf9a4f788447cc81f57f0f98d91e3d P cb66e0b3eac24a2f89d26b204ad4858a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:44:43.117651 20944 raft_consensus.cc:2258] T efaf9a4f788447cc81f57f0f98d91e3d P 68a1a25643ff4b9aad520e2d7a1f5008 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:44:43.118546 20943 ts_tablet_manager.cc:1905] T efaf9a4f788447cc81f57f0f98d91e3d P cb66e0b3eac24a2f89d26b204ad4858a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:44:43.118607 20942 ts_tablet_manager.cc:1905] T efaf9a4f788447cc81f57f0f98d91e3d P dd6c97f5a88b46478293f12b101f8c7c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:44:43.118892 20599 tablet_service.cc:1467] Processing CreateTablet for tablet deb70aa750ae4edeb564a3c40b28002a (DEFAULT_TABLE table=test-table [id=c03291d4b573404ebf47f77eca815a00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:44:43.119416 20386 tablet_service.cc:1467] Processing CreateTablet for tablet deb70aa750ae4edeb564a3c40b28002a (DEFAULT_TABLE table=test-table [id=c03291d4b573404ebf47f77eca815a00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:44:43.119429 20787 tablet_service.cc:1467] Processing CreateTablet for tablet deb70aa750ae4edeb564a3c40b28002a (DEFAULT_TABLE table=test-table [id=c03291d4b573404ebf47f77eca815a00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:44:43.119570 20787 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet deb70aa750ae4edeb564a3c40b28002a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:43.119578 20386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet deb70aa750ae4edeb564a3c40b28002a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:43.121290 20943 ts_tablet_manager.cc:1918] T efaf9a4f788447cc81f57f0f98d91e3d P cb66e0b3eac24a2f89d26b204ad4858a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:44:43.121417 20911 tablet_bootstrap.cc:492] T deb70aa750ae4edeb564a3c40b28002a P 68a1a25643ff4b9aad520e2d7a1f5008: Bootstrap starting.
I20260806 22:44:43.121450 20943 log.cc:1196] T efaf9a4f788447cc81f57f0f98d91e3d P cb66e0b3eac24a2f89d26b204ad4858a: Deleting WAL directory at /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/ts-1/wal/wals/efaf9a4f788447cc81f57f0f98d91e3d
I20260806 22:44:43.121505 20599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet deb70aa750ae4edeb564a3c40b28002a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:43.121968 20911 tablet_bootstrap.cc:654] T deb70aa750ae4edeb564a3c40b28002a P 68a1a25643ff4b9aad520e2d7a1f5008: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:43.122431 20904 tablet_bootstrap.cc:492] T deb70aa750ae4edeb564a3c40b28002a P dd6c97f5a88b46478293f12b101f8c7c: Bootstrap starting.
I20260806 22:44:43.122776 20911 tablet_bootstrap.cc:492] T deb70aa750ae4edeb564a3c40b28002a P 68a1a25643ff4b9aad520e2d7a1f5008: No bootstrap required, opened a new log
I20260806 22:44:43.122825 20911 ts_tablet_manager.cc:1397] T deb70aa750ae4edeb564a3c40b28002a P 68a1a25643ff4b9aad520e2d7a1f5008: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:44:43.123064 20904 tablet_bootstrap.cc:654] T deb70aa750ae4edeb564a3c40b28002a P dd6c97f5a88b46478293f12b101f8c7c: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:43.123077 20911 raft_consensus.cc:348] T deb70aa750ae4edeb564a3c40b28002a P 68a1a25643ff4b9aad520e2d7a1f5008 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68a1a25643ff4b9aad520e2d7a1f5008" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33651 } } peers { permanent_uuid: "cb66e0b3eac24a2f89d26b204ad4858a" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 42677 } } peers { permanent_uuid: "dd6c97f5a88b46478293f12b101f8c7c" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 38227 } }
I20260806 22:44:43.123178 20911 raft_consensus.cc:374] T deb70aa750ae4edeb564a3c40b28002a P 68a1a25643ff4b9aad520e2d7a1f5008 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:43.123202 20911 raft_consensus.cc:729] T deb70aa750ae4edeb564a3c40b28002a P 68a1a25643ff4b9aad520e2d7a1f5008 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68a1a25643ff4b9aad520e2d7a1f5008, State: Initialized, Role: FOLLOWER
I20260806 22:44:43.123246 20911 consensus_queue.cc:260] T deb70aa750ae4edeb564a3c40b28002a P 68a1a25643ff4b9aad520e2d7a1f5008 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68a1a25643ff4b9aad520e2d7a1f5008" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33651 } } peers { permanent_uuid: "cb66e0b3eac24a2f89d26b204ad4858a" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 42677 } } peers { permanent_uuid: "dd6c97f5a88b46478293f12b101f8c7c" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 38227 } }
I20260806 22:44:43.123278 20907 tablet_bootstrap.cc:492] T deb70aa750ae4edeb564a3c40b28002a P cb66e0b3eac24a2f89d26b204ad4858a: Bootstrap starting.
I20260806 22:44:43.123376 20911 ts_tablet_manager.cc:1428] T deb70aa750ae4edeb564a3c40b28002a P 68a1a25643ff4b9aad520e2d7a1f5008: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:44:43.123832 20907 tablet_bootstrap.cc:654] T deb70aa750ae4edeb564a3c40b28002a P cb66e0b3eac24a2f89d26b204ad4858a: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:43.124033 20944 ts_tablet_manager.cc:1905] T efaf9a4f788447cc81f57f0f98d91e3d P 68a1a25643ff4b9aad520e2d7a1f5008: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:44:43.124963 20943 ts_tablet_manager.cc:1939] T efaf9a4f788447cc81f57f0f98d91e3d P cb66e0b3eac24a2f89d26b204ad4858a: Deleting consensus metadata
I20260806 22:44:43.125487 20169 catalog_manager.cc:4841] TS cb66e0b3eac24a2f89d26b204ad4858a (127.18.45.194:42677): tablet efaf9a4f788447cc81f57f0f98d91e3d (table test-table [id=ab83a5ca350b496eb894b95a8d59dda0]) successfully deleted
I20260806 22:44:43.125840 20907 tablet_bootstrap.cc:492] T deb70aa750ae4edeb564a3c40b28002a P cb66e0b3eac24a2f89d26b204ad4858a: No bootstrap required, opened a new log
I20260806 22:44:43.125911 20907 ts_tablet_manager.cc:1397] T deb70aa750ae4edeb564a3c40b28002a P cb66e0b3eac24a2f89d26b204ad4858a: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 22:44:43.126161 20907 raft_consensus.cc:348] T deb70aa750ae4edeb564a3c40b28002a P cb66e0b3eac24a2f89d26b204ad4858a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68a1a25643ff4b9aad520e2d7a1f5008" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33651 } } peers { permanent_uuid: "cb66e0b3eac24a2f89d26b204ad4858a" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 42677 } } peers { permanent_uuid: "dd6c97f5a88b46478293f12b101f8c7c" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 38227 } }
I20260806 22:44:43.126255 20907 raft_consensus.cc:374] T deb70aa750ae4edeb564a3c40b28002a P cb66e0b3eac24a2f89d26b204ad4858a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:43.126276 20907 raft_consensus.cc:729] T deb70aa750ae4edeb564a3c40b28002a P cb66e0b3eac24a2f89d26b204ad4858a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb66e0b3eac24a2f89d26b204ad4858a, State: Initialized, Role: FOLLOWER
I20260806 22:44:43.126318 20907 consensus_queue.cc:260] T deb70aa750ae4edeb564a3c40b28002a P cb66e0b3eac24a2f89d26b204ad4858a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68a1a25643ff4b9aad520e2d7a1f5008" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33651 } } peers { permanent_uuid: "cb66e0b3eac24a2f89d26b204ad4858a" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 42677 } } peers { permanent_uuid: "dd6c97f5a88b46478293f12b101f8c7c" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 38227 } }
I20260806 22:44:43.126410 20907 ts_tablet_manager.cc:1428] T deb70aa750ae4edeb564a3c40b28002a P cb66e0b3eac24a2f89d26b204ad4858a: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 22:44:43.127590 20944 ts_tablet_manager.cc:1918] T efaf9a4f788447cc81f57f0f98d91e3d P 68a1a25643ff4b9aad520e2d7a1f5008: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:44:43.127607 20942 ts_tablet_manager.cc:1918] T efaf9a4f788447cc81f57f0f98d91e3d P dd6c97f5a88b46478293f12b101f8c7c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:44:43.127691 20942 log.cc:1196] T efaf9a4f788447cc81f57f0f98d91e3d P dd6c97f5a88b46478293f12b101f8c7c: Deleting WAL directory at /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/ts-2/wal/wals/efaf9a4f788447cc81f57f0f98d91e3d
I20260806 22:44:43.127753 20944 log.cc:1196] T efaf9a4f788447cc81f57f0f98d91e3d P 68a1a25643ff4b9aad520e2d7a1f5008: Deleting WAL directory at /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/ts-0/wal/wals/efaf9a4f788447cc81f57f0f98d91e3d
I20260806 22:44:43.128010 20942 ts_tablet_manager.cc:1939] T efaf9a4f788447cc81f57f0f98d91e3d P dd6c97f5a88b46478293f12b101f8c7c: Deleting consensus metadata
I20260806 22:44:43.128621 20167 catalog_manager.cc:4841] TS dd6c97f5a88b46478293f12b101f8c7c (127.18.45.195:38227): tablet efaf9a4f788447cc81f57f0f98d91e3d (table test-table [id=ab83a5ca350b496eb894b95a8d59dda0]) successfully deleted
I20260806 22:44:43.128764 20944 ts_tablet_manager.cc:1939] T efaf9a4f788447cc81f57f0f98d91e3d P 68a1a25643ff4b9aad520e2d7a1f5008: Deleting consensus metadata
I20260806 22:44:43.129256 20167 catalog_manager.cc:4841] TS 68a1a25643ff4b9aad520e2d7a1f5008 (127.18.45.193:33651): tablet efaf9a4f788447cc81f57f0f98d91e3d (table test-table [id=ab83a5ca350b496eb894b95a8d59dda0]) successfully deleted
I20260806 22:44:43.129547 20904 tablet_bootstrap.cc:492] T deb70aa750ae4edeb564a3c40b28002a P dd6c97f5a88b46478293f12b101f8c7c: No bootstrap required, opened a new log
I20260806 22:44:43.129616 20904 ts_tablet_manager.cc:1397] T deb70aa750ae4edeb564a3c40b28002a P dd6c97f5a88b46478293f12b101f8c7c: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.001s
I20260806 22:44:43.129801 20904 raft_consensus.cc:348] T deb70aa750ae4edeb564a3c40b28002a P dd6c97f5a88b46478293f12b101f8c7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68a1a25643ff4b9aad520e2d7a1f5008" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33651 } } peers { permanent_uuid: "cb66e0b3eac24a2f89d26b204ad4858a" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 42677 } } peers { permanent_uuid: "dd6c97f5a88b46478293f12b101f8c7c" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 38227 } }
I20260806 22:44:43.129890 20904 raft_consensus.cc:374] T deb70aa750ae4edeb564a3c40b28002a P dd6c97f5a88b46478293f12b101f8c7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:43.129915 20904 raft_consensus.cc:729] T deb70aa750ae4edeb564a3c40b28002a P dd6c97f5a88b46478293f12b101f8c7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd6c97f5a88b46478293f12b101f8c7c, State: Initialized, Role: FOLLOWER
I20260806 22:44:43.129978 20904 consensus_queue.cc:260] T deb70aa750ae4edeb564a3c40b28002a P dd6c97f5a88b46478293f12b101f8c7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68a1a25643ff4b9aad520e2d7a1f5008" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33651 } } peers { permanent_uuid: "cb66e0b3eac24a2f89d26b204ad4858a" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 42677 } } peers { permanent_uuid: "dd6c97f5a88b46478293f12b101f8c7c" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 38227 } }
I20260806 22:44:43.130088 20904 ts_tablet_manager.cc:1428] T deb70aa750ae4edeb564a3c40b28002a P dd6c97f5a88b46478293f12b101f8c7c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:43.147675 20939 raft_consensus.cc:482] T deb70aa750ae4edeb564a3c40b28002a P 68a1a25643ff4b9aad520e2d7a1f5008 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:44:43.147835 20939 raft_consensus.cc:504] T deb70aa750ae4edeb564a3c40b28002a P 68a1a25643ff4b9aad520e2d7a1f5008 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68a1a25643ff4b9aad520e2d7a1f5008" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33651 } } peers { permanent_uuid: "cb66e0b3eac24a2f89d26b204ad4858a" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 42677 } } peers { permanent_uuid: "dd6c97f5a88b46478293f12b101f8c7c" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 38227 } }
I20260806 22:44:43.148372 20939 leader_election.cc:288] T deb70aa750ae4edeb564a3c40b28002a P 68a1a25643ff4b9aad520e2d7a1f5008 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb66e0b3eac24a2f89d26b204ad4858a (127.18.45.194:42677), dd6c97f5a88b46478293f12b101f8c7c (127.18.45.195:38227)
I20260806 22:44:43.152189 20900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:43.148911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:36735 (local address 127.18.45.194:42677)
0806 22:44:43.148972 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:43.148974 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:43.148984 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:44:43.149050 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:43.149051 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:43.149065 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:43.149136 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:44:43.149370 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.149917 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.151260 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.151762 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.151850 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:43.151863 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:43.152048 (+ 185us) server_negotiation.cc:299] Negotiation successful
0806 22:44:43.152099 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 22:44:43.152201 20950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:43.148575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.194:42677 (local address 127.18.45.193:36735)
0806 22:44:43.148795 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:43.148811 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:43.148903 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:43.149157 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:43.149158 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:43.149161 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:44:43.149264 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:43.149267 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.150015 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.150016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:43.151155 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:43.151158 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.151878 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.151879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:43.151962 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:43.151967 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:44:43.152028 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 22:44:43.152078 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":138,"threads_started":1}
I20260806 22:44:43.152670 20899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:43.148763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:58549 (local address 127.18.45.195:38227)
0806 22:44:43.148823 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:43.148826 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:43.148944 (+ 118us) server_negotiation.cc:407] Connection header received
0806 22:44:43.149067 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:43.149068 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:43.149080 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:43.149141 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:44:43.149433 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.150204 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.151619 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.152176 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.152261 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:43.152272 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:43.152520 (+ 248us) server_negotiation.cc:299] Negotiation successful
0806 22:44:43.152588 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 22:44:43.152731 20623 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deb70aa750ae4edeb564a3c40b28002a" candidate_uuid: "68a1a25643ff4b9aad520e2d7a1f5008" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb66e0b3eac24a2f89d26b204ad4858a" is_pre_election: true
I20260806 22:44:43.152918 20623 raft_consensus.cc:2454] T deb70aa750ae4edeb564a3c40b28002a P cb66e0b3eac24a2f89d26b204ad4858a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68a1a25643ff4b9aad520e2d7a1f5008 in term 0.
I20260806 22:44:43.152946 20951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:43.148589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.195:38227 (local address 127.18.45.193:58549)
0806 22:44:43.148868 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:43.148880 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:43.148975 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:43.149202 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:43.149203 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:43.149205 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:44:43.149302 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:43.149335 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.150307 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.150308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:43.151521 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:43.151524 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.152316 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.152317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:43.152421 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:43.152425 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:44:43.152782 (+ 357us) client_negotiation.cc:240] Negotiation successful
0806 22:44:43.152840 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":118,"threads_started":1}
I20260806 22:44:43.153261 20320 leader_election.cc:302] T deb70aa750ae4edeb564a3c40b28002a P 68a1a25643ff4b9aad520e2d7a1f5008 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68a1a25643ff4b9aad520e2d7a1f5008, cb66e0b3eac24a2f89d26b204ad4858a; no voters:
I20260806 22:44:43.153288 20812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deb70aa750ae4edeb564a3c40b28002a" candidate_uuid: "68a1a25643ff4b9aad520e2d7a1f5008" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd6c97f5a88b46478293f12b101f8c7c" is_pre_election: true
I20260806 22:44:43.153427 20812 raft_consensus.cc:2454] T deb70aa750ae4edeb564a3c40b28002a P dd6c97f5a88b46478293f12b101f8c7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68a1a25643ff4b9aad520e2d7a1f5008 in term 0.
I20260806 22:44:43.153640 20939 raft_consensus.cc:2781] T deb70aa750ae4edeb564a3c40b28002a P 68a1a25643ff4b9aad520e2d7a1f5008 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:44:43.153729 20939 raft_consensus.cc:482] T deb70aa750ae4edeb564a3c40b28002a P 68a1a25643ff4b9aad520e2d7a1f5008 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:44:43.153770 20939 raft_consensus.cc:3037] T deb70aa750ae4edeb564a3c40b28002a P 68a1a25643ff4b9aad520e2d7a1f5008 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:43.154724 20939 raft_consensus.cc:504] T deb70aa750ae4edeb564a3c40b28002a P 68a1a25643ff4b9aad520e2d7a1f5008 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68a1a25643ff4b9aad520e2d7a1f5008" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33651 } } peers { permanent_uuid: "cb66e0b3eac24a2f89d26b204ad4858a" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 42677 } } peers { permanent_uuid: "dd6c97f5a88b46478293f12b101f8c7c" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 38227 } }
I20260806 22:44:43.154923 20939 leader_election.cc:288] T deb70aa750ae4edeb564a3c40b28002a P 68a1a25643ff4b9aad520e2d7a1f5008 [CANDIDATE]: Term 1 election: Requested vote from peers cb66e0b3eac24a2f89d26b204ad4858a (127.18.45.194:42677), dd6c97f5a88b46478293f12b101f8c7c (127.18.45.195:38227)
I20260806 22:44:43.155191 20623 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deb70aa750ae4edeb564a3c40b28002a" candidate_uuid: "68a1a25643ff4b9aad520e2d7a1f5008" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb66e0b3eac24a2f89d26b204ad4858a"
I20260806 22:44:43.155176 20812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deb70aa750ae4edeb564a3c40b28002a" candidate_uuid: "68a1a25643ff4b9aad520e2d7a1f5008" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd6c97f5a88b46478293f12b101f8c7c"
I20260806 22:44:43.155282 20623 raft_consensus.cc:3037] T deb70aa750ae4edeb564a3c40b28002a P cb66e0b3eac24a2f89d26b204ad4858a [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:43.155282 20812 raft_consensus.cc:3037] T deb70aa750ae4edeb564a3c40b28002a P dd6c97f5a88b46478293f12b101f8c7c [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:43.156363 20812 raft_consensus.cc:2454] T deb70aa750ae4edeb564a3c40b28002a P dd6c97f5a88b46478293f12b101f8c7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68a1a25643ff4b9aad520e2d7a1f5008 in term 1.
I20260806 22:44:43.156476 20623 raft_consensus.cc:2454] T deb70aa750ae4edeb564a3c40b28002a P cb66e0b3eac24a2f89d26b204ad4858a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68a1a25643ff4b9aad520e2d7a1f5008 in term 1.
I20260806 22:44:43.156584 20317 leader_election.cc:302] T deb70aa750ae4edeb564a3c40b28002a P 68a1a25643ff4b9aad520e2d7a1f5008 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68a1a25643ff4b9aad520e2d7a1f5008, dd6c97f5a88b46478293f12b101f8c7c; no voters:
I20260806 22:44:43.156751 20939 raft_consensus.cc:2781] T deb70aa750ae4edeb564a3c40b28002a P 68a1a25643ff4b9aad520e2d7a1f5008 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:44:43.156880 20939 raft_consensus.cc:686] T deb70aa750ae4edeb564a3c40b28002a P 68a1a25643ff4b9aad520e2d7a1f5008 [term 1 LEADER]: Becoming Leader. State: Replica: 68a1a25643ff4b9aad520e2d7a1f5008, State: Running, Role: LEADER
I20260806 22:44:43.156988 20939 consensus_queue.cc:237] T deb70aa750ae4edeb564a3c40b28002a P 68a1a25643ff4b9aad520e2d7a1f5008 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68a1a25643ff4b9aad520e2d7a1f5008" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33651 } } peers { permanent_uuid: "cb66e0b3eac24a2f89d26b204ad4858a" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 42677 } } peers { permanent_uuid: "dd6c97f5a88b46478293f12b101f8c7c" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 38227 } }
I20260806 22:44:43.158092 20189 catalog_manager.cc:5495] T deb70aa750ae4edeb564a3c40b28002a P 68a1a25643ff4b9aad520e2d7a1f5008 reported cstate change: term changed from 0 to 1, leader changed from <none> to 68a1a25643ff4b9aad520e2d7a1f5008 (127.18.45.193). New cstate: current_term: 1 leader_uuid: "68a1a25643ff4b9aad520e2d7a1f5008" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68a1a25643ff4b9aad520e2d7a1f5008" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33651 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb66e0b3eac24a2f89d26b204ad4858a" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 42677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd6c97f5a88b46478293f12b101f8c7c" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 38227 } health_report { overall_health: UNKNOWN } } }
Aug 06 22:44:43 dist-test-slave-h3ft krb5kdc[20125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056282, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.254@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:44:43.201701 20961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:43.191331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:42241 (local address 127.0.0.1:37652)
0806 22:44:43.191560 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:43.191573 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:43.191687 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:43.191923 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:43.191924 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:43.192512 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:43.192617 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:43.192623 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.193724 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.193725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:43.194701 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:43.194705 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.195251 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.195252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:43.195314 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:43.196618 (+ 1304us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:44:43.196624 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:43.198687 (+ 2063us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:43.200561 (+ 1874us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:43.200569 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:43.200571 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:43.200785 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:43.201020 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:43.201021 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:43.201021 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:43.201086 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:43.201390 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:43.201398 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:43.201445 (+ 47us) client_negotiation.cc:769] Sending connection context
0806 22:44:43.201499 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 22:44:43.201608 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":148,"threads_started":1}
I20260806 22:44:43.201803 20262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:43.191361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37652 (local address 127.18.45.254:42241)
0806 22:44:43.191420 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:43.191424 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:43.191650 (+ 226us) server_negotiation.cc:407] Connection header received
0806 22:44:43.191760 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:43.191761 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:43.191773 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:43.191820 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:43.192722 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.193626 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.194805 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.195148 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.195227 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:43.198806 (+ 3579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:43.198808 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:43.198809 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:43.198818 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:43.200467 (+ 1649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:43.200869 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:43.200871 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:43.200871 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:43.200945 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:43.201154 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:43.201155 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:43.201156 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:43.201311 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:43.201400 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:43.201559 (+ 159us) server_negotiation.cc:299] Negotiation successful
0806 22:44:43.201678 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":24}
I20260806 22:44:43.204638 20189 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:37652:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 22:44:43.205107 20189 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:44:43.207160 20599 tablet_service.cc:1467] Processing CreateTablet for tablet feb39d875977497cb25661517d050daa (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a5046228c7a24e8881dd9509a251c0fc]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:44:43.207275 20787 tablet_service.cc:1467] Processing CreateTablet for tablet feb39d875977497cb25661517d050daa (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a5046228c7a24e8881dd9509a251c0fc]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:44:43.207257 20386 tablet_service.cc:1467] Processing CreateTablet for tablet feb39d875977497cb25661517d050daa (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a5046228c7a24e8881dd9509a251c0fc]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:44:43.207450 20787 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet feb39d875977497cb25661517d050daa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:43.207450 20599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet feb39d875977497cb25661517d050daa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:43.207459 20386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet feb39d875977497cb25661517d050daa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:43.209338 20907 tablet_bootstrap.cc:492] T feb39d875977497cb25661517d050daa P cb66e0b3eac24a2f89d26b204ad4858a: Bootstrap starting.
I20260806 22:44:43.209338 20904 tablet_bootstrap.cc:492] T feb39d875977497cb25661517d050daa P dd6c97f5a88b46478293f12b101f8c7c: Bootstrap starting.
I20260806 22:44:43.209522 20911 tablet_bootstrap.cc:492] T feb39d875977497cb25661517d050daa P 68a1a25643ff4b9aad520e2d7a1f5008: Bootstrap starting.
I20260806 22:44:43.209954 20907 tablet_bootstrap.cc:654] T feb39d875977497cb25661517d050daa P cb66e0b3eac24a2f89d26b204ad4858a: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:43.210031 20904 tablet_bootstrap.cc:654] T feb39d875977497cb25661517d050daa P dd6c97f5a88b46478293f12b101f8c7c: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:43.210086 20911 tablet_bootstrap.cc:654] T feb39d875977497cb25661517d050daa P 68a1a25643ff4b9aad520e2d7a1f5008: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:43.211210 20907 tablet_bootstrap.cc:492] T feb39d875977497cb25661517d050daa P cb66e0b3eac24a2f89d26b204ad4858a: No bootstrap required, opened a new log
I20260806 22:44:43.211230 20904 tablet_bootstrap.cc:492] T feb39d875977497cb25661517d050daa P dd6c97f5a88b46478293f12b101f8c7c: No bootstrap required, opened a new log
I20260806 22:44:43.211269 20907 ts_tablet_manager.cc:1397] T feb39d875977497cb25661517d050daa P cb66e0b3eac24a2f89d26b204ad4858a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:44:43.211285 20904 ts_tablet_manager.cc:1397] T feb39d875977497cb25661517d050daa P dd6c97f5a88b46478293f12b101f8c7c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 22:44:43.211374 20911 tablet_bootstrap.cc:492] T feb39d875977497cb25661517d050daa P 68a1a25643ff4b9aad520e2d7a1f5008: No bootstrap required, opened a new log
I20260806 22:44:43.211424 20911 ts_tablet_manager.cc:1397] T feb39d875977497cb25661517d050daa P 68a1a25643ff4b9aad520e2d7a1f5008: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:44:43.211479 20907 raft_consensus.cc:348] T feb39d875977497cb25661517d050daa P cb66e0b3eac24a2f89d26b204ad4858a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68a1a25643ff4b9aad520e2d7a1f5008" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33651 } } peers { permanent_uuid: "cb66e0b3eac24a2f89d26b204ad4858a" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 42677 } } peers { permanent_uuid: "dd6c97f5a88b46478293f12b101f8c7c" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 38227 } }
I20260806 22:44:43.211521 20904 raft_consensus.cc:348] T feb39d875977497cb25661517d050daa P dd6c97f5a88b46478293f12b101f8c7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68a1a25643ff4b9aad520e2d7a1f5008" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33651 } } peers { permanent_uuid: "cb66e0b3eac24a2f89d26b204ad4858a" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 42677 } } peers { permanent_uuid: "dd6c97f5a88b46478293f12b101f8c7c" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 38227 } }
I20260806 22:44:43.211597 20907 raft_consensus.cc:374] T feb39d875977497cb25661517d050daa P cb66e0b3eac24a2f89d26b204ad4858a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:43.211601 20904 raft_consensus.cc:374] T feb39d875977497cb25661517d050daa P dd6c97f5a88b46478293f12b101f8c7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:43.211635 20907 raft_consensus.cc:729] T feb39d875977497cb25661517d050daa P cb66e0b3eac24a2f89d26b204ad4858a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb66e0b3eac24a2f89d26b204ad4858a, State: Initialized, Role: FOLLOWER
I20260806 22:44:43.211591 20911 raft_consensus.cc:348] T feb39d875977497cb25661517d050daa P 68a1a25643ff4b9aad520e2d7a1f5008 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68a1a25643ff4b9aad520e2d7a1f5008" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33651 } } peers { permanent_uuid: "cb66e0b3eac24a2f89d26b204ad4858a" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 42677 } } peers { permanent_uuid: "dd6c97f5a88b46478293f12b101f8c7c" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 38227 } }
I20260806 22:44:43.211651 20904 raft_consensus.cc:729] T feb39d875977497cb25661517d050daa P dd6c97f5a88b46478293f12b101f8c7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd6c97f5a88b46478293f12b101f8c7c, State: Initialized, Role: FOLLOWER
I20260806 22:44:43.211663 20911 raft_consensus.cc:374] T feb39d875977497cb25661517d050daa P 68a1a25643ff4b9aad520e2d7a1f5008 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:43.211689 20911 raft_consensus.cc:729] T feb39d875977497cb25661517d050daa P 68a1a25643ff4b9aad520e2d7a1f5008 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68a1a25643ff4b9aad520e2d7a1f5008, State: Initialized, Role: FOLLOWER
I20260806 22:44:43.211689 20907 consensus_queue.cc:260] T feb39d875977497cb25661517d050daa P cb66e0b3eac24a2f89d26b204ad4858a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68a1a25643ff4b9aad520e2d7a1f5008" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33651 } } peers { permanent_uuid: "cb66e0b3eac24a2f89d26b204ad4858a" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 42677 } } peers { permanent_uuid: "dd6c97f5a88b46478293f12b101f8c7c" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 38227 } }
I20260806 22:44:43.211709 20904 consensus_queue.cc:260] T feb39d875977497cb25661517d050daa P dd6c97f5a88b46478293f12b101f8c7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68a1a25643ff4b9aad520e2d7a1f5008" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33651 } } peers { permanent_uuid: "cb66e0b3eac24a2f89d26b204ad4858a" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 42677 } } peers { permanent_uuid: "dd6c97f5a88b46478293f12b101f8c7c" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 38227 } }
I20260806 22:44:43.211735 20911 consensus_queue.cc:260] T feb39d875977497cb25661517d050daa P 68a1a25643ff4b9aad520e2d7a1f5008 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68a1a25643ff4b9aad520e2d7a1f5008" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33651 } } peers { permanent_uuid: "cb66e0b3eac24a2f89d26b204ad4858a" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 42677 } } peers { permanent_uuid: "dd6c97f5a88b46478293f12b101f8c7c" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 38227 } }
I20260806 22:44:43.211802 20907 ts_tablet_manager.cc:1428] T feb39d875977497cb25661517d050daa P cb66e0b3eac24a2f89d26b204ad4858a: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260806 22:44:43.211817 20904 ts_tablet_manager.cc:1428] T feb39d875977497cb25661517d050daa P dd6c97f5a88b46478293f12b101f8c7c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:43.211829 20911 ts_tablet_manager.cc:1428] T feb39d875977497cb25661517d050daa P 68a1a25643ff4b9aad520e2d7a1f5008: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 22:44:43.212044 20928 tablet_replica.cc:439] T feb39d875977497cb25661517d050daa P cb66e0b3eac24a2f89d26b204ad4858a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68a1a25643ff4b9aad520e2d7a1f5008" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33651 } } peers { permanent_uuid: "cb66e0b3eac24a2f89d26b204ad4858a" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 42677 } } peers { permanent_uuid: "dd6c97f5a88b46478293f12b101f8c7c" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 38227 } } }
I20260806 22:44:43.212044 20909 tablet_replica.cc:439] T feb39d875977497cb25661517d050daa P dd6c97f5a88b46478293f12b101f8c7c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68a1a25643ff4b9aad520e2d7a1f5008" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33651 } } peers { permanent_uuid: "cb66e0b3eac24a2f89d26b204ad4858a" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 42677 } } peers { permanent_uuid: "dd6c97f5a88b46478293f12b101f8c7c" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 38227 } } }
I20260806 22:44:43.212141 20909 tablet_replica.cc:442] T feb39d875977497cb25661517d050daa P dd6c97f5a88b46478293f12b101f8c7c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:44:43.212141 20928 tablet_replica.cc:442] T feb39d875977497cb25661517d050daa P cb66e0b3eac24a2f89d26b204ad4858a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:44:43.212293 20914 tablet_replica.cc:439] T feb39d875977497cb25661517d050daa P 68a1a25643ff4b9aad520e2d7a1f5008: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68a1a25643ff4b9aad520e2d7a1f5008" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33651 } } peers { permanent_uuid: "cb66e0b3eac24a2f89d26b204ad4858a" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 42677 } } peers { permanent_uuid: "dd6c97f5a88b46478293f12b101f8c7c" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 38227 } } }
I20260806 22:44:43.212427 20914 tablet_replica.cc:442] T feb39d875977497cb25661517d050daa P 68a1a25643ff4b9aad520e2d7a1f5008: This TxnStatusTablet replica's current role is: FOLLOWER
W20260806 22:44:43.239939 20872 tablet.cc:2369] T deb70aa750ae4edeb564a3c40b28002a P dd6c97f5a88b46478293f12b101f8c7c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:44:43.288465 20690 tablet.cc:2369] T deb70aa750ae4edeb564a3c40b28002a P cb66e0b3eac24a2f89d26b204ad4858a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:44:43.593526 20914 raft_consensus.cc:482] T feb39d875977497cb25661517d050daa P 68a1a25643ff4b9aad520e2d7a1f5008 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:44:43.593688 20914 raft_consensus.cc:504] T feb39d875977497cb25661517d050daa P 68a1a25643ff4b9aad520e2d7a1f5008 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68a1a25643ff4b9aad520e2d7a1f5008" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33651 } } peers { permanent_uuid: "cb66e0b3eac24a2f89d26b204ad4858a" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 42677 } } peers { permanent_uuid: "dd6c97f5a88b46478293f12b101f8c7c" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 38227 } }
I20260806 22:44:43.593946 20914 leader_election.cc:288] T feb39d875977497cb25661517d050daa P 68a1a25643ff4b9aad520e2d7a1f5008 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb66e0b3eac24a2f89d26b204ad4858a (127.18.45.194:42677), dd6c97f5a88b46478293f12b101f8c7c (127.18.45.195:38227)
I20260806 22:44:43.594306 20623 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "feb39d875977497cb25661517d050daa" candidate_uuid: "68a1a25643ff4b9aad520e2d7a1f5008" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb66e0b3eac24a2f89d26b204ad4858a" is_pre_election: true
I20260806 22:44:43.594298 20812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "feb39d875977497cb25661517d050daa" candidate_uuid: "68a1a25643ff4b9aad520e2d7a1f5008" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd6c97f5a88b46478293f12b101f8c7c" is_pre_election: true
I20260806 22:44:43.594466 20812 raft_consensus.cc:2454] T feb39d875977497cb25661517d050daa P dd6c97f5a88b46478293f12b101f8c7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68a1a25643ff4b9aad520e2d7a1f5008 in term 0.
I20260806 22:44:43.594489 20623 raft_consensus.cc:2454] T feb39d875977497cb25661517d050daa P cb66e0b3eac24a2f89d26b204ad4858a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68a1a25643ff4b9aad520e2d7a1f5008 in term 0.
I20260806 22:44:43.594674 20317 leader_election.cc:302] T feb39d875977497cb25661517d050daa P 68a1a25643ff4b9aad520e2d7a1f5008 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68a1a25643ff4b9aad520e2d7a1f5008, dd6c97f5a88b46478293f12b101f8c7c; no voters:
I20260806 22:44:43.594806 20914 raft_consensus.cc:2781] T feb39d875977497cb25661517d050daa P 68a1a25643ff4b9aad520e2d7a1f5008 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:44:43.594890 20914 raft_consensus.cc:482] T feb39d875977497cb25661517d050daa P 68a1a25643ff4b9aad520e2d7a1f5008 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:44:43.594926 20914 raft_consensus.cc:3037] T feb39d875977497cb25661517d050daa P 68a1a25643ff4b9aad520e2d7a1f5008 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:43.596212 20914 raft_consensus.cc:504] T feb39d875977497cb25661517d050daa P 68a1a25643ff4b9aad520e2d7a1f5008 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68a1a25643ff4b9aad520e2d7a1f5008" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33651 } } peers { permanent_uuid: "cb66e0b3eac24a2f89d26b204ad4858a" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 42677 } } peers { permanent_uuid: "dd6c97f5a88b46478293f12b101f8c7c" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 38227 } }
I20260806 22:44:43.596400 20914 leader_election.cc:288] T feb39d875977497cb25661517d050daa P 68a1a25643ff4b9aad520e2d7a1f5008 [CANDIDATE]: Term 1 election: Requested vote from peers cb66e0b3eac24a2f89d26b204ad4858a (127.18.45.194:42677), dd6c97f5a88b46478293f12b101f8c7c (127.18.45.195:38227)
I20260806 22:44:43.596648 20812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "feb39d875977497cb25661517d050daa" candidate_uuid: "68a1a25643ff4b9aad520e2d7a1f5008" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd6c97f5a88b46478293f12b101f8c7c"
I20260806 22:44:43.596653 20623 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "feb39d875977497cb25661517d050daa" candidate_uuid: "68a1a25643ff4b9aad520e2d7a1f5008" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb66e0b3eac24a2f89d26b204ad4858a"
I20260806 22:44:43.596750 20812 raft_consensus.cc:3037] T feb39d875977497cb25661517d050daa P dd6c97f5a88b46478293f12b101f8c7c [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:43.596771 20623 raft_consensus.cc:3037] T feb39d875977497cb25661517d050daa P cb66e0b3eac24a2f89d26b204ad4858a [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:43.597836 20623 raft_consensus.cc:2454] T feb39d875977497cb25661517d050daa P cb66e0b3eac24a2f89d26b204ad4858a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68a1a25643ff4b9aad520e2d7a1f5008 in term 1.
I20260806 22:44:43.597851 20812 raft_consensus.cc:2454] T feb39d875977497cb25661517d050daa P dd6c97f5a88b46478293f12b101f8c7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68a1a25643ff4b9aad520e2d7a1f5008 in term 1.
I20260806 22:44:43.598075 20320 leader_election.cc:302] T feb39d875977497cb25661517d050daa P 68a1a25643ff4b9aad520e2d7a1f5008 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68a1a25643ff4b9aad520e2d7a1f5008, cb66e0b3eac24a2f89d26b204ad4858a; no voters:
I20260806 22:44:43.598196 20914 raft_consensus.cc:2781] T feb39d875977497cb25661517d050daa P 68a1a25643ff4b9aad520e2d7a1f5008 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:44:43.598273 20914 raft_consensus.cc:686] T feb39d875977497cb25661517d050daa P 68a1a25643ff4b9aad520e2d7a1f5008 [term 1 LEADER]: Becoming Leader. State: Replica: 68a1a25643ff4b9aad520e2d7a1f5008, State: Running, Role: LEADER
I20260806 22:44:43.598344 20914 consensus_queue.cc:237] T feb39d875977497cb25661517d050daa P 68a1a25643ff4b9aad520e2d7a1f5008 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68a1a25643ff4b9aad520e2d7a1f5008" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33651 } } peers { permanent_uuid: "cb66e0b3eac24a2f89d26b204ad4858a" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 42677 } } peers { permanent_uuid: "dd6c97f5a88b46478293f12b101f8c7c" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 38227 } }
I20260806 22:44:43.598714 20939 tablet_replica.cc:439] T feb39d875977497cb25661517d050daa P 68a1a25643ff4b9aad520e2d7a1f5008: TxnStatusTablet state changed. Reason: New leader 68a1a25643ff4b9aad520e2d7a1f5008. Latest consensus state: current_term: 1 leader_uuid: "68a1a25643ff4b9aad520e2d7a1f5008" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68a1a25643ff4b9aad520e2d7a1f5008" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33651 } } peers { permanent_uuid: "cb66e0b3eac24a2f89d26b204ad4858a" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 42677 } } peers { permanent_uuid: "dd6c97f5a88b46478293f12b101f8c7c" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 38227 } } }
I20260806 22:44:43.598821 20939 tablet_replica.cc:442] T feb39d875977497cb25661517d050daa P 68a1a25643ff4b9aad520e2d7a1f5008: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:44:43.599066 20968 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 22:44:43.599246 20968 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 22:44:43.599200 20189 catalog_manager.cc:5495] T feb39d875977497cb25661517d050daa P 68a1a25643ff4b9aad520e2d7a1f5008 reported cstate change: term changed from 0 to 1, leader changed from <none> to 68a1a25643ff4b9aad520e2d7a1f5008 (127.18.45.193). New cstate: current_term: 1 leader_uuid: "68a1a25643ff4b9aad520e2d7a1f5008" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68a1a25643ff4b9aad520e2d7a1f5008" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33651 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb66e0b3eac24a2f89d26b204ad4858a" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 42677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd6c97f5a88b46478293f12b101f8c7c" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 38227 } health_report { overall_health: UNKNOWN } } }
I20260806 22:44:43.656306 20939 consensus_queue.cc:1035] T deb70aa750ae4edeb564a3c40b28002a P 68a1a25643ff4b9aad520e2d7a1f5008 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb66e0b3eac24a2f89d26b204ad4858a" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 42677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:44:43.658797 20939 consensus_queue.cc:1035] T deb70aa750ae4edeb564a3c40b28002a P 68a1a25643ff4b9aad520e2d7a1f5008 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd6c97f5a88b46478293f12b101f8c7c" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 38227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:44:43.659934 20961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:43.656030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.193:33651 (local address 127.0.0.1:46252)
0806 22:44:43.656130 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:43.656144 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:43.656204 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:43.656494 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:43.656495 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:43.656496 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:44:43.656571 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:43.656575 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.657703 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.657705 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:43.658631 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:43.658637 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.659175 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.659176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:43.659264 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:43.659289 (+ 25us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:44:43.659678 (+ 389us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:44:43.659693 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 22:44:43.659744 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 22:44:43.659819 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260806 22:44:43.660219 20974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:43.656076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46252 (local address 127.18.45.193:33651)
0806 22:44:43.656318 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:43.656323 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:43.656336 (+ 13us) server_negotiation.cc:407] Connection header received
0806 22:44:43.656367 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:43.656368 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:43.656394 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:43.656486 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:44:43.656747 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.657585 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.658745 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.659094 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.659158 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:43.659434 (+ 276us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:44:43.659586 (+ 152us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:44:43.659695 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:43.660111 (+ 416us) server_negotiation.cc:299] Negotiation successful
0806 22:44:43.660144 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":61,"threads_started":1}
I20260806 22:44:43.661358 20812 raft_consensus.cc:1261] T feb39d875977497cb25661517d050daa P dd6c97f5a88b46478293f12b101f8c7c [term 1 FOLLOWER]: Refusing update from remote peer 68a1a25643ff4b9aad520e2d7a1f5008: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:44:43.661427 20623 raft_consensus.cc:1261] T feb39d875977497cb25661517d050daa P cb66e0b3eac24a2f89d26b204ad4858a [term 1 FOLLOWER]: Refusing update from remote peer 68a1a25643ff4b9aad520e2d7a1f5008: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:44:43.661688 20939 consensus_queue.cc:1035] T feb39d875977497cb25661517d050daa P 68a1a25643ff4b9aad520e2d7a1f5008 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd6c97f5a88b46478293f12b101f8c7c" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 38227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:44:43.661794 20914 consensus_queue.cc:1035] T feb39d875977497cb25661517d050daa P 68a1a25643ff4b9aad520e2d7a1f5008 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb66e0b3eac24a2f89d26b204ad4858a" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 42677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:44:43.662377 20928 tablet_replica.cc:439] T feb39d875977497cb25661517d050daa P cb66e0b3eac24a2f89d26b204ad4858a: TxnStatusTablet state changed. Reason: New leader 68a1a25643ff4b9aad520e2d7a1f5008. Latest consensus state: current_term: 1 leader_uuid: "68a1a25643ff4b9aad520e2d7a1f5008" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68a1a25643ff4b9aad520e2d7a1f5008" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33651 } } peers { permanent_uuid: "cb66e0b3eac24a2f89d26b204ad4858a" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 42677 } } peers { permanent_uuid: "dd6c97f5a88b46478293f12b101f8c7c" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 38227 } } }
I20260806 22:44:43.662503 20928 tablet_replica.cc:442] T feb39d875977497cb25661517d050daa P cb66e0b3eac24a2f89d26b204ad4858a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:44:43.662631 20980 tablet_replica.cc:439] T feb39d875977497cb25661517d050daa P dd6c97f5a88b46478293f12b101f8c7c: TxnStatusTablet state changed. Reason: New leader 68a1a25643ff4b9aad520e2d7a1f5008. Latest consensus state: current_term: 1 leader_uuid: "68a1a25643ff4b9aad520e2d7a1f5008" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68a1a25643ff4b9aad520e2d7a1f5008" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33651 } } peers { permanent_uuid: "cb66e0b3eac24a2f89d26b204ad4858a" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 42677 } } peers { permanent_uuid: "dd6c97f5a88b46478293f12b101f8c7c" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 38227 } } }
I20260806 22:44:43.662750 20980 tablet_replica.cc:442] T feb39d875977497cb25661517d050daa P dd6c97f5a88b46478293f12b101f8c7c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:44:43.663625 20914 tablet_replica.cc:439] T feb39d875977497cb25661517d050daa P 68a1a25643ff4b9aad520e2d7a1f5008: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "68a1a25643ff4b9aad520e2d7a1f5008" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68a1a25643ff4b9aad520e2d7a1f5008" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33651 } } peers { permanent_uuid: "cb66e0b3eac24a2f89d26b204ad4858a" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 42677 } } peers { permanent_uuid: "dd6c97f5a88b46478293f12b101f8c7c" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 38227 } } }
I20260806 22:44:43.663753 20914 tablet_replica.cc:442] T feb39d875977497cb25661517d050daa P 68a1a25643ff4b9aad520e2d7a1f5008: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:44:43.663734 20939 tablet_replica.cc:439] T feb39d875977497cb25661517d050daa P 68a1a25643ff4b9aad520e2d7a1f5008: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "68a1a25643ff4b9aad520e2d7a1f5008" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68a1a25643ff4b9aad520e2d7a1f5008" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33651 } } peers { permanent_uuid: "cb66e0b3eac24a2f89d26b204ad4858a" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 42677 } } peers { permanent_uuid: "dd6c97f5a88b46478293f12b101f8c7c" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 38227 } } }
I20260806 22:44:43.663798 20939 tablet_replica.cc:442] T feb39d875977497cb25661517d050daa P 68a1a25643ff4b9aad520e2d7a1f5008: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:44:43.663854 20977 mvcc.cc:205] Tried to move back new op lower bound from 7315686537875427328 to 7315686537619505152. Current Snapshot: MvccSnapshot[applied={T|T < 7315686537875427328}]
I20260806 22:44:43.663949 20928 tablet_replica.cc:439] T feb39d875977497cb25661517d050daa P cb66e0b3eac24a2f89d26b204ad4858a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "68a1a25643ff4b9aad520e2d7a1f5008" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68a1a25643ff4b9aad520e2d7a1f5008" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33651 } } peers { permanent_uuid: "cb66e0b3eac24a2f89d26b204ad4858a" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 42677 } } peers { permanent_uuid: "dd6c97f5a88b46478293f12b101f8c7c" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 38227 } } }
I20260806 22:44:43.664055 20928 tablet_replica.cc:442] T feb39d875977497cb25661517d050daa P cb66e0b3eac24a2f89d26b204ad4858a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:44:43.664536 20980 tablet_replica.cc:439] T feb39d875977497cb25661517d050daa P dd6c97f5a88b46478293f12b101f8c7c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "68a1a25643ff4b9aad520e2d7a1f5008" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68a1a25643ff4b9aad520e2d7a1f5008" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33651 } } peers { permanent_uuid: "cb66e0b3eac24a2f89d26b204ad4858a" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 42677 } } peers { permanent_uuid: "dd6c97f5a88b46478293f12b101f8c7c" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 38227 } } }
I20260806 22:44:43.664636 20980 tablet_replica.cc:442] T feb39d875977497cb25661517d050daa P dd6c97f5a88b46478293f12b101f8c7c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:44:43.663973 20976 mvcc.cc:205] Tried to move back new op lower bound from 7315686537875427328 to 7315686537619505152. Current Snapshot: MvccSnapshot[applied={T|T < 7315686537875427328}]
I20260806 22:44:43.672406 20974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:43.669476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46254 (local address 127.18.45.193:33651)
0806 22:44:43.669534 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:43.669537 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:43.669636 (+ 99us) server_negotiation.cc:407] Connection header received
0806 22:44:43.669713 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:43.669714 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:43.669728 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:43.669801 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:44:43.670043 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.670647 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.671642 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.671863 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.671952 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:43.672121 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:44:43.672164 (+ 43us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:44:43.672220 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:43.672281 (+ 61us) server_negotiation.cc:299] Negotiation successful
0806 22:44:43.672308 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 22:44:43.678582 20974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:43.675625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:47877 (local address 127.18.45.193:33651)
0806 22:44:43.675692 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:43.675694 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:43.675836 (+ 142us) server_negotiation.cc:407] Connection header received
0806 22:44:43.675898 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:43.675899 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:43.675913 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:43.675992 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:44:43.676144 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.676795 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.677670 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.677950 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.678041 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:43.678267 (+ 226us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:44:43.678304 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:44:43.678368 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:43.678457 (+ 89us) server_negotiation.cc:299] Negotiation successful
0806 22:44:43.678484 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 22:44:43.679342 20996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:43.675553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.193:33651 (local address 127.18.45.193:47877)
0806 22:44:43.675755 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:43.675766 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:43.675836 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:43.675984 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:43.675985 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:43.675987 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:44:43.676080 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:43.676082 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.676906 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.676907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:43.677572 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:43.677574 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.678091 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.678092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:43.678158 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:43.678174 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:44:43.678397 (+ 223us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:44:43.678400 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:44:43.678453 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 22:44:43.678500 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":99,"threads_started":1}
I20260806 22:44:43.697896 20187 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37648:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:44:43.698081 20187 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37648:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:44:43.699059 20187 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ece890efc8674a71b7ee1a70429fc1bb: Sending DeleteTablet for 3 replicas of tablet deb70aa750ae4edeb564a3c40b28002a
I20260806 22:44:43.699416 20599 tablet_service.cc:1514] Processing DeleteTablet for tablet deb70aa750ae4edeb564a3c40b28002a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:44:43 UTC) from {username='kudu', principal='kudu/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:36320
I20260806 22:44:43.699473 20386 tablet_service.cc:1514] Processing DeleteTablet for tablet deb70aa750ae4edeb564a3c40b28002a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:44:43 UTC) from {username='kudu', principal='kudu/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:46242
I20260806 22:44:43.699571 20787 tablet_service.cc:1514] Processing DeleteTablet for tablet deb70aa750ae4edeb564a3c40b28002a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:44:43 UTC) from {username='kudu', principal='kudu/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:58096
I20260806 22:44:43.699813 21000 tablet_replica.cc:330] T deb70aa750ae4edeb564a3c40b28002a P 68a1a25643ff4b9aad520e2d7a1f5008: stopping tablet replica
I20260806 22:44:43.699910 21000 raft_consensus.cc:2229] T deb70aa750ae4edeb564a3c40b28002a P 68a1a25643ff4b9aad520e2d7a1f5008 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:44:43.700040 21000 raft_consensus.cc:2258] T deb70aa750ae4edeb564a3c40b28002a P 68a1a25643ff4b9aad520e2d7a1f5008 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:44:43.700177 20999 tablet_replica.cc:330] T deb70aa750ae4edeb564a3c40b28002a P cb66e0b3eac24a2f89d26b204ad4858a: stopping tablet replica
I20260806 22:44:43.700248 20999 raft_consensus.cc:2229] T deb70aa750ae4edeb564a3c40b28002a P cb66e0b3eac24a2f89d26b204ad4858a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:44:43.700297 20999 raft_consensus.cc:2258] T deb70aa750ae4edeb564a3c40b28002a P cb66e0b3eac24a2f89d26b204ad4858a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:44:43.700562 21000 ts_tablet_manager.cc:1905] T deb70aa750ae4edeb564a3c40b28002a P 68a1a25643ff4b9aad520e2d7a1f5008: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:44:43.700600 20999 ts_tablet_manager.cc:1905] T deb70aa750ae4edeb564a3c40b28002a P cb66e0b3eac24a2f89d26b204ad4858a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:44:43.700858 21001 tablet_replica.cc:330] T deb70aa750ae4edeb564a3c40b28002a P dd6c97f5a88b46478293f12b101f8c7c: stopping tablet replica
I20260806 22:44:43.700965 21001 raft_consensus.cc:2229] T deb70aa750ae4edeb564a3c40b28002a P dd6c97f5a88b46478293f12b101f8c7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:44:43.701030 21001 raft_consensus.cc:2258] T deb70aa750ae4edeb564a3c40b28002a P dd6c97f5a88b46478293f12b101f8c7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:44:43.701368 21001 ts_tablet_manager.cc:1905] T deb70aa750ae4edeb564a3c40b28002a P dd6c97f5a88b46478293f12b101f8c7c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:44:43.703074 20999 ts_tablet_manager.cc:1918] T deb70aa750ae4edeb564a3c40b28002a P cb66e0b3eac24a2f89d26b204ad4858a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:44:43.703075 21000 ts_tablet_manager.cc:1918] T deb70aa750ae4edeb564a3c40b28002a P 68a1a25643ff4b9aad520e2d7a1f5008: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:44:43.703141 20999 log.cc:1196] T deb70aa750ae4edeb564a3c40b28002a P cb66e0b3eac24a2f89d26b204ad4858a: Deleting WAL directory at /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/ts-1/wal/wals/deb70aa750ae4edeb564a3c40b28002a
I20260806 22:44:43.703141 21000 log.cc:1196] T deb70aa750ae4edeb564a3c40b28002a P 68a1a25643ff4b9aad520e2d7a1f5008: Deleting WAL directory at /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/ts-0/wal/wals/deb70aa750ae4edeb564a3c40b28002a
I20260806 22:44:43.703408 21000 ts_tablet_manager.cc:1939] T deb70aa750ae4edeb564a3c40b28002a P 68a1a25643ff4b9aad520e2d7a1f5008: Deleting consensus metadata
I20260806 22:44:43.703418 20999 ts_tablet_manager.cc:1939] T deb70aa750ae4edeb564a3c40b28002a P cb66e0b3eac24a2f89d26b204ad4858a: Deleting consensus metadata
I20260806 22:44:43.703743 21001 ts_tablet_manager.cc:1918] T deb70aa750ae4edeb564a3c40b28002a P dd6c97f5a88b46478293f12b101f8c7c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:44:43.703821 21001 log.cc:1196] T deb70aa750ae4edeb564a3c40b28002a P dd6c97f5a88b46478293f12b101f8c7c: Deleting WAL directory at /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786056275803658-18615-0/minicluster-data/ts-2/wal/wals/deb70aa750ae4edeb564a3c40b28002a
I20260806 22:44:43.703897 20167 catalog_manager.cc:4841] TS 68a1a25643ff4b9aad520e2d7a1f5008 (127.18.45.193:33651): tablet deb70aa750ae4edeb564a3c40b28002a (table test-table [id=c03291d4b573404ebf47f77eca815a00]) successfully deleted
I20260806 22:44:43.704028 20169 catalog_manager.cc:4841] TS cb66e0b3eac24a2f89d26b204ad4858a (127.18.45.194:42677): tablet deb70aa750ae4edeb564a3c40b28002a (table test-table [id=c03291d4b573404ebf47f77eca815a00]) successfully deleted
I20260806 22:44:43.704304 21001 ts_tablet_manager.cc:1939] T deb70aa750ae4edeb564a3c40b28002a P dd6c97f5a88b46478293f12b101f8c7c: Deleting consensus metadata
I20260806 22:44:43.704699 20167 catalog_manager.cc:4841] TS dd6c97f5a88b46478293f12b101f8c7c (127.18.45.195:38227): tablet deb70aa750ae4edeb564a3c40b28002a (table test-table [id=c03291d4b573404ebf47f77eca815a00]) successfully deleted
Aug 06 22:44:43 dist-test-slave-h3ft krb5kdc[20125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056283, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.45.193@KRBTEST.COM
I20260806 22:44:43.715294 20974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:43.704137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46262 (local address 127.18.45.193:33651)
0806 22:44:43.704193 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:43.704196 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:43.704456 (+ 260us) server_negotiation.cc:407] Connection header received
0806 22:44:43.704530 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:43.704531 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:43.704545 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:43.704605 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:43.705308 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.705837 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.706635 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.706890 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.706983 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:43.710848 (+ 3865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:43.710859 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:43.710859 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:43.710870 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:43.713653 (+ 2783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:43.714161 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:43.714167 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:43.714167 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:43.714272 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:43.714539 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:43.714540 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:43.714540 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:43.714753 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:43.714945 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:43.715035 (+ 90us) server_negotiation.cc:299] Negotiation successful
0806 22:44:43.715155 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
W20260806 22:44:43.715775 20450 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46262
I20260806 22:44:43.724753 21006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:43.717259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37654 (local address 127.18.45.254:42241)
0806 22:44:43.717410 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:43.717413 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:43.717422 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:44:43.717455 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:43.717456 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:43.717472 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:43.717571 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:43.718222 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.718830 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.719809 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:43.720057 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:43.720138 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:43.721726 (+ 1588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:43.721729 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:43.721729 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:43.721739 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:43.723415 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:43.723812 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:43.723814 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:43.723814 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:43.723889 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:43.724104 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:43.724105 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:43.724106 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:43.724241 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:43.724377 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:43.724513 (+ 136us) server_negotiation.cc:299] Negotiation successful
0806 22:44:43.724612 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":72,"threads_started":1}
W20260806 22:44:43.725061 20187 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37654
I20260806 22:44:43.725811 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 20290
I20260806 22:44:43.743494 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 20492
I20260806 22:44:43.761770 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 20700
I20260806 22:44:43.776427 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 20143
2026-08-06T22:44:43Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2977 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:44:43 dist-test-slave-h3ft krb5kdc[21009](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:44:43 dist-test-slave-h3ft krb5kdc[21009](info): set up 2 sockets
Aug 06 22:44:43 dist-test-slave-h3ft krb5kdc[21009](info): commencing operation
krb5kdc: starting...
W20260806 22:44:45.042248 18615 mini_kdc.cc:121] Time spent starting KDC: real 1.232s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Aug 06 22:44:45 dist-test-slave-h3ft krb5kdc[21009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:44:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:44:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.45.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab.
Entry for principal kudu/127.18.45.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab.
I20260806 22:44:45.190789 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:37773
--webserver_interface=127.18.45.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:41671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.45.254:37773
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:44:45.287159 21026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:45.287489 21026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:45.287530 21026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:45.288793 21026 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:44:45.288885 21026 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 22:44:45.288913 21026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:45.288946 21026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:45.288980 21026 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:44:45.289009 21026 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:44:45.290692 21026 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:41671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.45.254:37773
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:37773
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.45.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21026
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:44:45.291078 21026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:45.291384 21026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:45.297328 21040 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:45.297328 21036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:45.297385 21035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:45.297708 21026 server_base.cc:1034] running on GCE node
I20260806 22:44:45.298046 21026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:45.298377 21026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:45.299538 21026 hybrid_clock.cc:648] HybridClock initialized: now 1786056285299529 us; error 40 us; skew 500 ppm
Aug 06 22:44:45 dist-test-slave-h3ft krb5kdc[21009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056285, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:45.302904 21026 init.cc:377] Logged in from keytab as kudu/127.18.45.254@KRBTEST.COM (short username kudu)
I20260806 22:44:45.303563 21026 webserver.cc:460] Webserver started at http://127.18.45.254:37217/ using document root <none> and password file <none>
I20260806 22:44:45.303846 21026 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:45.303920 21026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:45.304098 21026 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:44:45.305404 21026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/master-0/data/instance:
uuid: "e54ba47edd124325be88cc0cc4df70b5"
format_stamp: "Formatted at 2026-08-06 22:44:45 on dist-test-slave-h3ft"
server_key: "cba678c791f7ae0a9994162b182ea0e5"
server_key_iv: "a6f119f9a4a7683e233347743b9337d2"
server_key_version: "encryptionkey@0"
I20260806 22:44:45.305852 21026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/master-0/wal/instance:
uuid: "e54ba47edd124325be88cc0cc4df70b5"
format_stamp: "Formatted at 2026-08-06 22:44:45 on dist-test-slave-h3ft"
server_key: "cba678c791f7ae0a9994162b182ea0e5"
server_key_iv: "a6f119f9a4a7683e233347743b9337d2"
server_key_version: "encryptionkey@0"
I20260806 22:44:45.307477 21026 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260806 22:44:45.308429 21048 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:45.308697 21026 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:44:45.308827 21026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/master-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/master-0/wal
uuid: "e54ba47edd124325be88cc0cc4df70b5"
format_stamp: "Formatted at 2026-08-06 22:44:45 on dist-test-slave-h3ft"
server_key: "cba678c791f7ae0a9994162b182ea0e5"
server_key_iv: "a6f119f9a4a7683e233347743b9337d2"
server_key_version: "encryptionkey@0"
I20260806 22:44:45.308923 21026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:45.323321 21026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:45.325867 21026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:45.326087 21026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:45.333089 21026 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.254:37773
I20260806 22:44:45.333160 21134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.254:37773 every 8 connection(s)
I20260806 22:44:45.333576 21026 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
I20260806 22:44:45.334422 21135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:45.337235 21135 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e54ba47edd124325be88cc0cc4df70b5: Bootstrap starting.
I20260806 22:44:45.337961 21135 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e54ba47edd124325be88cc0cc4df70b5: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:45.338357 21135 log.cc:824] T 00000000000000000000000000000000 P e54ba47edd124325be88cc0cc4df70b5: Log is configured to *not* fsync() on all Append() calls
I20260806 22:44:45.338398 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 21026
I20260806 22:44:45.338491 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/master-0/wal/instance
I20260806 22:44:45.338719 18615 external_mini_cluster.cc:1428] Setting key e18c52edbbdd8420b3be3c0132048acf
I20260806 22:44:45.339356 21135 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e54ba47edd124325be88cc0cc4df70b5: No bootstrap required, opened a new log
I20260806 22:44:45.342557 21135 raft_consensus.cc:348] T 00000000000000000000000000000000 P e54ba47edd124325be88cc0cc4df70b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e54ba47edd124325be88cc0cc4df70b5" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 37773 } }
I20260806 22:44:45.342751 21135 raft_consensus.cc:374] T 00000000000000000000000000000000 P e54ba47edd124325be88cc0cc4df70b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:45.342801 21135 raft_consensus.cc:729] T 00000000000000000000000000000000 P e54ba47edd124325be88cc0cc4df70b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e54ba47edd124325be88cc0cc4df70b5, State: Initialized, Role: FOLLOWER
I20260806 22:44:45.343006 21135 consensus_queue.cc:260] T 00000000000000000000000000000000 P e54ba47edd124325be88cc0cc4df70b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e54ba47edd124325be88cc0cc4df70b5" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 37773 } }
I20260806 22:44:45.343168 21135 raft_consensus.cc:388] T 00000000000000000000000000000000 P e54ba47edd124325be88cc0cc4df70b5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:44:45.343216 21135 raft_consensus.cc:482] T 00000000000000000000000000000000 P e54ba47edd124325be88cc0cc4df70b5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:44:45.343272 21135 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e54ba47edd124325be88cc0cc4df70b5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:45.344198 21135 raft_consensus.cc:504] T 00000000000000000000000000000000 P e54ba47edd124325be88cc0cc4df70b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e54ba47edd124325be88cc0cc4df70b5" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 37773 } }
I20260806 22:44:45.344324 21135 leader_election.cc:302] T 00000000000000000000000000000000 P e54ba47edd124325be88cc0cc4df70b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e54ba47edd124325be88cc0cc4df70b5; no voters:
I20260806 22:44:45.344520 21135 leader_election.cc:288] T 00000000000000000000000000000000 P e54ba47edd124325be88cc0cc4df70b5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:44:45.344573 21141 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e54ba47edd124325be88cc0cc4df70b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:44:45.344765 21141 raft_consensus.cc:686] T 00000000000000000000000000000000 P e54ba47edd124325be88cc0cc4df70b5 [term 1 LEADER]: Becoming Leader. State: Replica: e54ba47edd124325be88cc0cc4df70b5, State: Running, Role: LEADER
I20260806 22:44:45.344847 21135 sys_catalog.cc:564] T 00000000000000000000000000000000 P e54ba47edd124325be88cc0cc4df70b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:44:45.344924 21141 consensus_queue.cc:237] T 00000000000000000000000000000000 P e54ba47edd124325be88cc0cc4df70b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e54ba47edd124325be88cc0cc4df70b5" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 37773 } }
I20260806 22:44:45.345350 21141 sys_catalog.cc:455] T 00000000000000000000000000000000 P e54ba47edd124325be88cc0cc4df70b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e54ba47edd124325be88cc0cc4df70b5. Latest consensus state: current_term: 1 leader_uuid: "e54ba47edd124325be88cc0cc4df70b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e54ba47edd124325be88cc0cc4df70b5" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 37773 } } }
I20260806 22:44:45.345443 21141 sys_catalog.cc:458] T 00000000000000000000000000000000 P e54ba47edd124325be88cc0cc4df70b5 [sys.catalog]: This master's current role is: LEADER
I20260806 22:44:45.345582 21142 sys_catalog.cc:455] T 00000000000000000000000000000000 P e54ba47edd124325be88cc0cc4df70b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e54ba47edd124325be88cc0cc4df70b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e54ba47edd124325be88cc0cc4df70b5" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 37773 } } }
I20260806 22:44:45.345750 21142 sys_catalog.cc:458] T 00000000000000000000000000000000 P e54ba47edd124325be88cc0cc4df70b5 [sys.catalog]: This master's current role is: LEADER
I20260806 22:44:45.345772 21147 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:44:45.346619 21147 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 22:44:45 dist-test-slave-h3ft krb5kdc[21009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:44:45.348635 21147 catalog_manager.cc:1347] Generated new cluster ID: 1bbafbf99a3244f6baf21e42c4e3e3bc
I20260806 22:44:45.348693 21147 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:44:45.353096 21139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:45.339495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47954 (local address 127.18.45.254:37773)
0806 22:44:45.339698 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:45.339707 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:45.339738 (+ 31us) server_negotiation.cc:407] Connection header received
0806 22:44:45.340813 (+ 1075us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:45.340822 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:45.340882 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:45.341031 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:45.342040 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:45.343341 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:45.344535 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:45.345264 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:45.346032 (+ 768us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:45.348946 (+ 2914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:45.348951 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:45.348951 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:45.348966 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:45.351164 (+ 2198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:45.351635 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:45.351639 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:45.351640 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:45.351755 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:45.351991 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:45.351992 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:45.351992 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:45.352317 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:45.352482 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:45.352700 (+ 218us) server_negotiation.cc:299] Negotiation successful
0806 22:44:45.352849 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":97,"threads_started":1}
I20260806 22:44:45.375583 21147 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:44:45.376245 21147 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:44:45.383260 21147 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e54ba47edd124325be88cc0cc4df70b5: Generated new TSK 0
I20260806 22:44:45.383558 21147 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.45.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab.
Entry for principal kudu/127.18.45.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab.
I20260806 22:44:45.422909 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.193:0
--local_ip_for_outbound_sockets=127.18.45.193
--webserver_interface=127.18.45.193
--webserver_port=0
--tserver_master_addrs=127.18.45.254:37773
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab
--principal=kudu/127.18.45.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:41671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:44:45.520676 21164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:45.520932 21164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:45.520970 21164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:45.522132 21164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:45.522168 21164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:45.522188 21164 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 22:44:45.522200 21164 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:44:45.522217 21164 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.193
I20260806 22:44:45.523718 21164 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:41671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab
--principal=kudu/127.18.45.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.45.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.45.254:37773
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21164
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.193
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:44:45.523970 21164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:45.524192 21164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:45.530647 21173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:45.530704 21176 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:45.530781 21174 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:45.531076 21164 server_base.cc:1034] running on GCE node
I20260806 22:44:45.531315 21164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:45.531603 21164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:45.532773 21164 hybrid_clock.cc:648] HybridClock initialized: now 1786056285532730 us; error 61 us; skew 500 ppm
Aug 06 22:44:45 dist-test-slave-h3ft krb5kdc[21009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056285, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:45.535974 21164 init.cc:377] Logged in from keytab as kudu/127.18.45.193@KRBTEST.COM (short username kudu)
I20260806 22:44:45.536549 21164 webserver.cc:460] Webserver started at http://127.18.45.193:38309/ using document root <none> and password file <none>
I20260806 22:44:45.536773 21164 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:45.536818 21164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:45.536923 21164 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:44:45.538166 21164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/ts-0/data/instance:
uuid: "92eb5ec5ac944cfca7c5420a72b4652c"
format_stamp: "Formatted at 2026-08-06 22:44:45 on dist-test-slave-h3ft"
server_key: "2236496bc9fe55987361788b877625cb"
server_key_iv: "0fd57200dda6c34c1f173c0fde74c69c"
server_key_version: "encryptionkey@0"
I20260806 22:44:45.538527 21164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/ts-0/wal/instance:
uuid: "92eb5ec5ac944cfca7c5420a72b4652c"
format_stamp: "Formatted at 2026-08-06 22:44:45 on dist-test-slave-h3ft"
server_key: "2236496bc9fe55987361788b877625cb"
server_key_iv: "0fd57200dda6c34c1f173c0fde74c69c"
server_key_version: "encryptionkey@0"
I20260806 22:44:45.540184 21164 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:44:45.541043 21186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:45.541289 21164 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:44:45.541414 21164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/ts-0/wal
uuid: "92eb5ec5ac944cfca7c5420a72b4652c"
format_stamp: "Formatted at 2026-08-06 22:44:45 on dist-test-slave-h3ft"
server_key: "2236496bc9fe55987361788b877625cb"
server_key_iv: "0fd57200dda6c34c1f173c0fde74c69c"
server_key_version: "encryptionkey@0"
I20260806 22:44:45.541486 21164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:45.551194 21164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:45.553890 21164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:45.554040 21164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:45.554672 21164 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:44:45.554719 21164 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:45.554747 21164 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:44:45.554804 21164 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 22:44:45 dist-test-slave-h3ft krb5kdc[21009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056285, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.193@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:44:45.564105 21164 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.193:33257
I20260806 22:44:45.564215 21353 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.193:33257 every 8 connection(s)
I20260806 22:44:45.564651 21164 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
I20260806 22:44:45.567174 21139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:45.555291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:43953 (local address 127.18.45.254:37773)
0806 22:44:45.555377 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:45.555381 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:45.555622 (+ 241us) server_negotiation.cc:407] Connection header received
0806 22:44:45.556396 (+ 774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:45.556398 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:45.556417 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:45.556518 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:45.557464 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:45.558408 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:45.560237 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:45.560613 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:45.560718 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:45.563966 (+ 3248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:45.563968 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:45.563968 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:45.563978 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:45.565752 (+ 1774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:45.566180 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:45.566180 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:45.566181 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:45.566251 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:45.566491 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:45.566491 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:45.566492 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:45.566622 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:45.566684 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:45.566920 (+ 236us) server_negotiation.cc:299] Negotiation successful
0806 22:44:45.567080 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 22:44:45.567312 21198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:45.555254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:37773 (local address 127.18.45.193:43953)
0806 22:44:45.555447 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:45.555494 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:45.556266 (+ 772us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:45.556582 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:45.556586 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:45.556966 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:45.557302 (+ 336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:45.557316 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:45.558529 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:45.558530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:45.560097 (+ 1567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:45.560101 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:45.560739 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:45.560740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:45.560926 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:45.561629 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:44:45.561658 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:45.563842 (+ 2184us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:45.565847 (+ 2005us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:45.565854 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:45.565856 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:45.566103 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:45.566318 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:45.566319 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:45.566319 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:45.566431 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:45.566676 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:45.566679 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:45.566805 (+ 126us) client_negotiation.cc:769] Sending connection context
0806 22:44:45.566890 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 22:44:45.567097 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":104,"threads_started":1}
I20260806 22:44:45.568197 21354 heartbeater.cc:347] Connected to a master server at 127.18.45.254:37773
I20260806 22:44:45.568334 21354 heartbeater.cc:464] Registering TS with master...
I20260806 22:44:45.568562 21354 heartbeater.cc:511] Master 127.18.45.254:37773 requested a full tablet report, sending...
I20260806 22:44:45.569303 21068 ts_manager.cc:194] Registered new tserver with Master: 92eb5ec5ac944cfca7c5420a72b4652c (127.18.45.193:33257)
I20260806 22:44:45.570365 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 21164
I20260806 22:44:45.570372 21068 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.45.193@KRBTEST.COM'} at 127.18.45.193:43953
I20260806 22:44:45.570508 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/ts-0/wal/instance
I20260806 22:44:45.570755 18615 external_mini_cluster.cc:1428] Setting key 081c6341e3d47fb2594b52a1ad5c0fe1
I20260806 22:44:45.576340 21362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:45.569505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:37773 (local address 127.18.45.193:52791)
0806 22:44:45.569676 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:45.569687 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:45.569744 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:45.569892 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:45.569893 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:45.570065 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:45.570125 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:45.570126 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:45.571169 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:45.571170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:45.571878 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:45.571880 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:45.572295 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:45.572296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:45.572396 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:45.572907 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:44:45.572913 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:45.573246 (+ 333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:45.575304 (+ 2058us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:45.575305 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:45.575305 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:45.575494 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:45.575728 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:45.575729 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:45.575729 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:45.575787 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:45.576090 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:45.576091 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:45.576132 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 22:44:45.576185 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 22:44:45.576266 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":114,"threads_started":1}
I20260806 22:44:45.576455 21139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:45.569502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:52791 (local address 127.18.45.254:37773)
0806 22:44:45.569545 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:45.569546 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:45.569786 (+ 240us) server_negotiation.cc:407] Connection header received
0806 22:44:45.569812 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:45.569813 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:45.569826 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:45.569892 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:45.570219 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:45.571003 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:45.571988 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:45.572211 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:45.572297 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:45.573368 (+ 1071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:45.573370 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:45.573371 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:45.573390 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:45.575200 (+ 1810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:45.575583 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:45.575584 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:45.575585 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:45.575657 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:45.575860 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:45.575861 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:45.575861 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:45.576024 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:45.576100 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:45.576218 (+ 118us) server_negotiation.cc:299] Negotiation successful
0806 22:44:45.576310 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
WARNING: no policy specified for kudu/127.18.45.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab.
Entry for principal kudu/127.18.45.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab.
I20260806 22:44:45.605715 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.194:0
--local_ip_for_outbound_sockets=127.18.45.194
--webserver_interface=127.18.45.194
--webserver_port=0
--tserver_master_addrs=127.18.45.254:37773
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab
--principal=kudu/127.18.45.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:41671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:44:45.705065 21365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:45.705367 21365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:45.705406 21365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:45.706838 21365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:45.706899 21365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:45.706929 21365 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 22:44:45.706945 21365 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:44:45.706965 21365 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.194
I20260806 22:44:45.708833 21365 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:41671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab
--principal=kudu/127.18.45.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.45.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.45.254:37773
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21365
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.194
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:44:45.709164 21365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:45.709484 21365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:45.715754 21375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:45.715847 21372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:45.716041 21365 server_base.cc:1034] running on GCE node
W20260806 22:44:45.715766 21373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:45.716329 21365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:45.716589 21365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:45.717792 21365 hybrid_clock.cc:648] HybridClock initialized: now 1786056285717767 us; error 42 us; skew 500 ppm
Aug 06 22:44:45 dist-test-slave-h3ft krb5kdc[21009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056285, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:45.721164 21365 init.cc:377] Logged in from keytab as kudu/127.18.45.194@KRBTEST.COM (short username kudu)
I20260806 22:44:45.721699 21365 webserver.cc:460] Webserver started at http://127.18.45.194:37905/ using document root <none> and password file <none>
I20260806 22:44:45.721967 21365 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:45.722043 21365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:45.722210 21365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:44:45.723618 21365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/ts-1/data/instance:
uuid: "af47fa363a39460f9128a46c2a601ec4"
format_stamp: "Formatted at 2026-08-06 22:44:45 on dist-test-slave-h3ft"
server_key: "8821c76a887679437134225b52c73bc4"
server_key_iv: "eda034a81117b9e406ad34d6acf211cf"
server_key_version: "encryptionkey@0"
I20260806 22:44:45.724048 21365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/ts-1/wal/instance:
uuid: "af47fa363a39460f9128a46c2a601ec4"
format_stamp: "Formatted at 2026-08-06 22:44:45 on dist-test-slave-h3ft"
server_key: "8821c76a887679437134225b52c73bc4"
server_key_iv: "eda034a81117b9e406ad34d6acf211cf"
server_key_version: "encryptionkey@0"
I20260806 22:44:45.725620 21365 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:44:45.726526 21389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:45.726755 21365 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:44:45.726876 21365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/ts-1/wal
uuid: "af47fa363a39460f9128a46c2a601ec4"
format_stamp: "Formatted at 2026-08-06 22:44:45 on dist-test-slave-h3ft"
server_key: "8821c76a887679437134225b52c73bc4"
server_key_iv: "eda034a81117b9e406ad34d6acf211cf"
server_key_version: "encryptionkey@0"
I20260806 22:44:45.726958 21365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:45.748931 21365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:45.752130 21365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:45.752368 21365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:45.753247 21365 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:44:45.753325 21365 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:45.753377 21365 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:44:45.753405 21365 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 22:44:45 dist-test-slave-h3ft krb5kdc[21009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056285, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.194@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:44:45.762095 21365 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.194:45227
I20260806 22:44:45.762149 21553 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.194:45227 every 8 connection(s)
I20260806 22:44:45.762612 21365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
I20260806 22:44:45.763442 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 21365
I20260806 22:44:45.763537 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/ts-1/wal/instance
I20260806 22:44:45.763778 18615 external_mini_cluster.cc:1428] Setting key a20bed40a25c53695b1e087178ed11ee
I20260806 22:44:45.766122 21139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:45.753897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:59933 (local address 127.18.45.254:37773)
0806 22:44:45.754001 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:45.754005 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:45.754264 (+ 259us) server_negotiation.cc:407] Connection header received
0806 22:44:45.754940 (+ 676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:45.754942 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:45.754960 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:45.755032 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:45.756115 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:45.757046 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:45.758838 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:45.759296 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:45.759413 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:45.762594 (+ 3181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:45.762596 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:45.762596 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:45.762609 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:45.764457 (+ 1848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:45.764949 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:45.764951 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:45.764952 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:45.765053 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:45.765277 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:45.765278 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:45.765279 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:45.765450 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:45.765521 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:45.765855 (+ 334us) server_negotiation.cc:299] Negotiation successful
0806 22:44:45.765977 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 22:44:45.766160 21401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:45.753870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:37773 (local address 127.18.45.194:59933)
0806 22:44:45.754085 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:45.754135 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:45.754812 (+ 677us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:45.755128 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:45.755132 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:45.755604 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:45.755997 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:45.756004 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:45.757142 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:45.757144 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:45.758685 (+ 1541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:45.758689 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:45.759470 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:45.759471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:45.759664 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:45.760433 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:44:45.760447 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:45.762454 (+ 2007us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:45.764530 (+ 2076us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:45.764539 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:45.764541 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:45.764862 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:45.765118 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:45.765119 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:45.765119 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:45.765219 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:45.765528 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:45.765534 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:45.765682 (+ 148us) client_negotiation.cc:769] Sending connection context
0806 22:44:45.765775 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 22:44:45.765931 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":119,"threads_started":1}
I20260806 22:44:45.766788 21554 heartbeater.cc:347] Connected to a master server at 127.18.45.254:37773
I20260806 22:44:45.766937 21554 heartbeater.cc:464] Registering TS with master...
I20260806 22:44:45.767294 21554 heartbeater.cc:511] Master 127.18.45.254:37773 requested a full tablet report, sending...
I20260806 22:44:45.768004 21068 ts_manager.cc:194] Registered new tserver with Master: af47fa363a39460f9128a46c2a601ec4 (127.18.45.194:45227)
I20260806 22:44:45.768613 21068 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.45.194@KRBTEST.COM'} at 127.18.45.194:59933
WARNING: no policy specified for kudu/127.18.45.195@KRBTEST.COM; defaulting to no policy
I20260806 22:44:45.777395 21139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:45.768221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:53145 (local address 127.18.45.254:37773)
0806 22:44:45.768273 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:45.768275 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:45.768473 (+ 198us) server_negotiation.cc:407] Connection header received
0806 22:44:45.768530 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:45.768532 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:45.768546 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:45.768600 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:45.769311 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:45.770066 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:45.772211 (+ 2145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:45.772610 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:45.772707 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:45.773980 (+ 1273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:45.773981 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:45.773982 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:45.773991 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:45.775921 (+ 1930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:45.776334 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:45.776335 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:45.776336 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:45.776409 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:45.776620 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:45.776622 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:45.776622 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:45.776894 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:45.776976 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:45.777167 (+ 191us) server_negotiation.cc:299] Negotiation successful
0806 22:44:45.777284 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 22:44:45.777391 21563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:45.768139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:37773 (local address 127.18.45.194:53145)
0806 22:44:45.768376 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:45.768388 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:45.768468 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:45.768613 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:45.768614 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:45.769123 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:45.769191 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:45.769193 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:45.771132 (+ 1939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:45.771133 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:45.772068 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:45.772071 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:45.772733 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:45.772734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:45.772815 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:45.773454 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:44:45.773460 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:45.773877 (+ 417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:45.776016 (+ 2139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:45.776017 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:45.776039 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:45.776248 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:45.776491 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:45.776491 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:45.776492 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:45.776555 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:45.776971 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:45.776972 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:45.777042 (+ 70us) client_negotiation.cc:769] Sending connection context
0806 22:44:45.777103 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 22:44:45.777212 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":164,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab.
Entry for principal kudu/127.18.45.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab.
I20260806 22:44:45.803592 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.195:0
--local_ip_for_outbound_sockets=127.18.45.195
--webserver_interface=127.18.45.195
--webserver_port=0
--tserver_master_addrs=127.18.45.254:37773
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab
--principal=kudu/127.18.45.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:41671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:44:45.902233 21565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:45.902483 21565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:45.902511 21565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:45.903753 21565 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:45.903797 21565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:45.903818 21565 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 22:44:45.903831 21565 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:44:45.903846 21565 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.195
I20260806 22:44:45.905369 21565 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:41671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab
--principal=kudu/127.18.45.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.45.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.45.254:37773
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21565
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.195
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:44:45.905606 21565 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:45.905822 21565 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:45.911619 21574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:45.911628 21576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:45.911630 21573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:45.912029 21565 server_base.cc:1034] running on GCE node
I20260806 22:44:45.912266 21565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:45.912531 21565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:45.913712 21565 hybrid_clock.cc:648] HybridClock initialized: now 1786056285913695 us; error 32 us; skew 500 ppm
Aug 06 22:44:45 dist-test-slave-h3ft krb5kdc[21009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056285, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:45.916688 21565 init.cc:377] Logged in from keytab as kudu/127.18.45.195@KRBTEST.COM (short username kudu)
I20260806 22:44:45.917204 21565 webserver.cc:460] Webserver started at http://127.18.45.195:35501/ using document root <none> and password file <none>
I20260806 22:44:45.917430 21565 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:45.917475 21565 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:45.917577 21565 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:44:45.918735 21565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/ts-2/data/instance:
uuid: "b711413b9ffb4c8b9072671dd5d5f199"
format_stamp: "Formatted at 2026-08-06 22:44:45 on dist-test-slave-h3ft"
server_key: "a5853e740f98e830b3028a14e6bd85c4"
server_key_iv: "9400fc93ff02a737d7994eb3a13f465b"
server_key_version: "encryptionkey@0"
I20260806 22:44:45.919229 21565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/ts-2/wal/instance:
uuid: "b711413b9ffb4c8b9072671dd5d5f199"
format_stamp: "Formatted at 2026-08-06 22:44:45 on dist-test-slave-h3ft"
server_key: "a5853e740f98e830b3028a14e6bd85c4"
server_key_iv: "9400fc93ff02a737d7994eb3a13f465b"
server_key_version: "encryptionkey@0"
I20260806 22:44:45.920674 21565 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:44:45.921541 21585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:45.921727 21565 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:44:45.921820 21565 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/ts-2/wal
uuid: "b711413b9ffb4c8b9072671dd5d5f199"
format_stamp: "Formatted at 2026-08-06 22:44:45 on dist-test-slave-h3ft"
server_key: "a5853e740f98e830b3028a14e6bd85c4"
server_key_iv: "9400fc93ff02a737d7994eb3a13f465b"
server_key_version: "encryptionkey@0"
I20260806 22:44:45.921876 21565 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:45.939445 21565 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:45.942009 21565 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:45.942149 21565 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:45.942730 21565 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:44:45.942787 21565 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:45.942812 21565 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:44:45.942826 21565 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 22:44:45 dist-test-slave-h3ft krb5kdc[21009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056285, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.195@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:44:45.951426 21565 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.195:44247
I20260806 22:44:45.951520 21752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.195:44247 every 8 connection(s)
I20260806 22:44:45.951964 21565 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
I20260806 22:44:45.955112 21139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:45.943357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.195:49623 (local address 127.18.45.254:37773)
0806 22:44:45.943444 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:45.943448 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:45.943674 (+ 226us) server_negotiation.cc:407] Connection header received
0806 22:44:45.944395 (+ 721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:45.944397 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:45.944414 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:45.944482 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:45.945347 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:45.946255 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:45.947954 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:45.948322 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:45.948434 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:45.951842 (+ 3408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:45.951844 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:45.951845 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:45.951855 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:45.953412 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:45.953942 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:45.953943 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:45.953944 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:45.954018 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:45.954353 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:45.954355 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:45.954355 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:45.954513 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:45.954586 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:45.954860 (+ 274us) server_negotiation.cc:299] Negotiation successful
0806 22:44:45.954945 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 22:44:45.955278 21598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:45.943309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:37773 (local address 127.18.45.195:49623)
0806 22:44:45.943519 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:45.943566 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:45.944212 (+ 646us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:45.944568 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:45.944573 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:45.944898 (+ 325us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:45.945243 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:45.945251 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:45.946350 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:45.946351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:45.947845 (+ 1494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:45.947849 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:45.948490 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:45.948491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:45.948658 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:45.949368 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:44:45.949379 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:45.951700 (+ 2321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:45.953505 (+ 1805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:45.953513 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:45.953515 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:45.953802 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:45.954119 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:45.954120 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:45.954120 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:45.954259 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:45.954583 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:45.954589 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:45.954733 (+ 144us) client_negotiation.cc:769] Sending connection context
0806 22:44:45.954826 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 22:44:45.954989 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":127,"threads_started":1}
I20260806 22:44:45.955893 21753 heartbeater.cc:347] Connected to a master server at 127.18.45.254:37773
I20260806 22:44:45.955992 21753 heartbeater.cc:464] Registering TS with master...
I20260806 22:44:45.956223 21753 heartbeater.cc:511] Master 127.18.45.254:37773 requested a full tablet report, sending...
I20260806 22:44:45.956631 21068 ts_manager.cc:194] Registered new tserver with Master: b711413b9ffb4c8b9072671dd5d5f199 (127.18.45.195:44247)
I20260806 22:44:45.957086 21068 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.45.195@KRBTEST.COM'} at 127.18.45.195:49623
I20260806 22:44:45.961531 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 21565
I20260806 22:44:45.961663 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/ts-2/wal/instance
I20260806 22:44:45.961918 18615 external_mini_cluster.cc:1428] Setting key 8faf145e25b2c21a9928a03ecc97afee
I20260806 22:44:45.963131 18615 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:44:45.963788 21139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:45.957259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.195:39965 (local address 127.18.45.254:37773)
0806 22:44:45.957301 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:45.957303 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:45.957471 (+ 168us) server_negotiation.cc:407] Connection header received
0806 22:44:45.957539 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:45.957540 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:45.957552 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:45.957594 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:45.958009 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:45.958499 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:45.959417 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:45.959612 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:45.959672 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:45.960691 (+ 1019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:45.960693 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:45.960693 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:45.960699 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:45.962408 (+ 1709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:45.962896 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:45.962898 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:45.962898 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:45.962977 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:45.963222 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:45.963223 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:45.963224 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:45.963351 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:45.963427 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:45.963597 (+ 170us) server_negotiation.cc:299] Negotiation successful
0806 22:44:45.963684 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:44:45.963791 21765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:45.957206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:37773 (local address 127.18.45.195:39965)
0806 22:44:45.957406 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:45.957417 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:45.957485 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:45.957606 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:45.957607 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:45.957837 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:45.957907 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:45.957909 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:45.958631 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:45.958632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:45.959332 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:45.959335 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:45.959706 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:45.959707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:45.959778 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:45.960281 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:44:45.960286 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:45.960599 (+ 313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:45.962491 (+ 1892us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:45.962492 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:45.962493 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:45.962800 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:45.963079 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:45.963080 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:45.963080 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:45.963146 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:45.963428 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:45.963430 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:45.963480 (+ 50us) client_negotiation.cc:769] Sending connection context
0806 22:44:45.963566 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 22:44:45.963686 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":145,"threads_started":1}
Aug 06 22:44:45 dist-test-slave-h3ft krb5kdc[21009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 22:44:45 dist-test-slave-h3ft krb5kdc[21009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:44:45.990851 21139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:45.982831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47968 (local address 127.18.45.254:37773)
0806 22:44:45.982924 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:45.982929 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:45.982944 (+ 15us) server_negotiation.cc:407] Connection header received
0806 22:44:45.982978 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:45.982979 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:45.982992 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:45.983127 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:45.983817 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:45.984387 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:45.985198 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:45.985415 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:45.985486 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:45.988061 (+ 2575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:45.988063 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:45.988063 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:45.988072 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:45.989703 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:45.990075 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:45.990076 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:45.990077 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:45.990146 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:45.990372 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:45.990373 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:45.990373 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:45.990515 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:45.990601 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:45.990693 (+ 92us) server_negotiation.cc:299] Negotiation successful
0806 22:44:45.990764 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 22:44:45.993043 21068 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47968:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:44:45.995150 21068 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:44:46.002693 21783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:45.998429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40484 (local address 127.18.45.194:45227)
0806 22:44:45.998700 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:45.998707 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:45.998721 (+ 14us) server_negotiation.cc:407] Connection header received
0806 22:44:45.998776 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:45.998779 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:45.998801 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:45.998894 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:44:45.999147 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:46.000051 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:46.001430 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:46.002215 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:46.002320 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:46.002403 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:46.002528 (+ 125us) server_negotiation.cc:299] Negotiation successful
0806 22:44:46.002578 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":149,"threads_started":1}
I20260806 22:44:46.002682 21777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:45.998132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.194:45227 (local address 127.0.0.1:40484)
0806 22:44:45.998334 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:45.998389 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:45.998473 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:45.998953 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:45.998956 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:45.998959 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:44:45.999075 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:45.999078 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:46.000145 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:46.000146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:46.001331 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:46.001333 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:46.002308 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:46.002309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:46.002396 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:46.002413 (+ 17us) client_negotiation.cc:769] Sending connection context
0806 22:44:46.002499 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 22:44:46.002570 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":114,"threads_started":1}
I20260806 22:44:46.003114 21780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:45.998137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.195:44247 (local address 127.0.0.1:46594)
0806 22:44:45.998716 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:45.998727 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:45.998791 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:45.999299 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:45.999300 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:45.999302 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:44:45.999360 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:45.999362 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:46.000725 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:46.000726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:46.001709 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:46.001712 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:46.002830 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:46.002832 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:46.002898 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:46.002903 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:44:46.002952 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 22:44:46.003030 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":124,"threads_started":1}
I20260806 22:44:46.003228 21779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:45.998132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.193:33257 (local address 127.0.0.1:47084)
0806 22:44:45.998603 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:45.998613 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:45.998930 (+ 317us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:45.999324 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:45.999325 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:45.999327 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:44:45.999392 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:45.999394 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:46.000837 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:46.000839 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:46.001884 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:46.001887 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:46.002887 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:46.002889 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:46.002961 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:46.002966 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:44:46.003074 (+ 108us) client_negotiation.cc:240] Negotiation successful
0806 22:44:46.003124 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"mutex_wait_us":2,"thread_start_us":52,"threads_started":1}
I20260806 22:44:46.003294 21781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:45.998385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47084 (local address 127.18.45.193:33257)
0806 22:44:45.998903 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:45.998908 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:45.998924 (+ 16us) server_negotiation.cc:407] Connection header received
0806 22:44:45.999195 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:45.999198 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:45.999219 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:45.999486 (+ 267us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:44:45.999581 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:46.000666 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:46.001976 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:46.002781 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:46.002871 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:46.002949 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:46.003090 (+ 141us) server_negotiation.cc:299] Negotiation successful
0806 22:44:46.003153 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":91,"threads_started":1}
I20260806 22:44:46.003479 21784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:45.998719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46594 (local address 127.18.45.195:44247)
0806 22:44:45.999110 (+ 391us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:45.999114 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:45.999125 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:44:45.999174 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:45.999177 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:45.999194 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:45.999291 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:44:45.999426 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:46.000623 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:46.001809 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:46.002641 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:46.002727 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:46.002792 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:46.003303 (+ 511us) server_negotiation.cc:299] Negotiation successful
0806 22:44:46.003358 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":112,"threads_started":1}
I20260806 22:44:46.004302 21640 tablet_service.cc:1467] Processing CreateTablet for tablet ca8de8e89fbf418bbc2ab79a53b448fe (DEFAULT_TABLE table=test-table [id=a60c867d5f064f9493512807af072e15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:44:46.004298 21260 tablet_service.cc:1467] Processing CreateTablet for tablet ca8de8e89fbf418bbc2ab79a53b448fe (DEFAULT_TABLE table=test-table [id=a60c867d5f064f9493512807af072e15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:44:46.004631 21464 tablet_service.cc:1467] Processing CreateTablet for tablet ca8de8e89fbf418bbc2ab79a53b448fe (DEFAULT_TABLE table=test-table [id=a60c867d5f064f9493512807af072e15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:44:46.004802 21260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca8de8e89fbf418bbc2ab79a53b448fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:46.004787 21640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca8de8e89fbf418bbc2ab79a53b448fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:46.005012 21464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca8de8e89fbf418bbc2ab79a53b448fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:46.008149 21785 tablet_bootstrap.cc:492] T ca8de8e89fbf418bbc2ab79a53b448fe P b711413b9ffb4c8b9072671dd5d5f199: Bootstrap starting.
I20260806 22:44:46.009268 21785 tablet_bootstrap.cc:654] T ca8de8e89fbf418bbc2ab79a53b448fe P b711413b9ffb4c8b9072671dd5d5f199: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:46.009447 21786 tablet_bootstrap.cc:492] T ca8de8e89fbf418bbc2ab79a53b448fe P 92eb5ec5ac944cfca7c5420a72b4652c: Bootstrap starting.
I20260806 22:44:46.009696 21785 log.cc:824] T ca8de8e89fbf418bbc2ab79a53b448fe P b711413b9ffb4c8b9072671dd5d5f199: Log is configured to *not* fsync() on all Append() calls
I20260806 22:44:46.010072 21787 tablet_bootstrap.cc:492] T ca8de8e89fbf418bbc2ab79a53b448fe P af47fa363a39460f9128a46c2a601ec4: Bootstrap starting.
I20260806 22:44:46.010176 21786 tablet_bootstrap.cc:654] T ca8de8e89fbf418bbc2ab79a53b448fe P 92eb5ec5ac944cfca7c5420a72b4652c: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:46.010601 21786 log.cc:824] T ca8de8e89fbf418bbc2ab79a53b448fe P 92eb5ec5ac944cfca7c5420a72b4652c: Log is configured to *not* fsync() on all Append() calls
I20260806 22:44:46.010664 21785 tablet_bootstrap.cc:492] T ca8de8e89fbf418bbc2ab79a53b448fe P b711413b9ffb4c8b9072671dd5d5f199: No bootstrap required, opened a new log
I20260806 22:44:46.010728 21785 ts_tablet_manager.cc:1397] T ca8de8e89fbf418bbc2ab79a53b448fe P b711413b9ffb4c8b9072671dd5d5f199: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:44:46.010895 21787 tablet_bootstrap.cc:654] T ca8de8e89fbf418bbc2ab79a53b448fe P af47fa363a39460f9128a46c2a601ec4: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:46.011319 21787 log.cc:824] T ca8de8e89fbf418bbc2ab79a53b448fe P af47fa363a39460f9128a46c2a601ec4: Log is configured to *not* fsync() on all Append() calls
I20260806 22:44:46.011468 21786 tablet_bootstrap.cc:492] T ca8de8e89fbf418bbc2ab79a53b448fe P 92eb5ec5ac944cfca7c5420a72b4652c: No bootstrap required, opened a new log
I20260806 22:44:46.011546 21786 ts_tablet_manager.cc:1397] T ca8de8e89fbf418bbc2ab79a53b448fe P 92eb5ec5ac944cfca7c5420a72b4652c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:44:46.012125 21787 tablet_bootstrap.cc:492] T ca8de8e89fbf418bbc2ab79a53b448fe P af47fa363a39460f9128a46c2a601ec4: No bootstrap required, opened a new log
I20260806 22:44:46.012217 21787 ts_tablet_manager.cc:1397] T ca8de8e89fbf418bbc2ab79a53b448fe P af47fa363a39460f9128a46c2a601ec4: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260806 22:44:46.013340 21785 raft_consensus.cc:348] T ca8de8e89fbf418bbc2ab79a53b448fe P b711413b9ffb4c8b9072671dd5d5f199 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33257 } } peers { permanent_uuid: "b711413b9ffb4c8b9072671dd5d5f199" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44247 } } peers { permanent_uuid: "af47fa363a39460f9128a46c2a601ec4" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45227 } }
I20260806 22:44:46.013559 21785 raft_consensus.cc:374] T ca8de8e89fbf418bbc2ab79a53b448fe P b711413b9ffb4c8b9072671dd5d5f199 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:46.013608 21785 raft_consensus.cc:729] T ca8de8e89fbf418bbc2ab79a53b448fe P b711413b9ffb4c8b9072671dd5d5f199 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b711413b9ffb4c8b9072671dd5d5f199, State: Initialized, Role: FOLLOWER
I20260806 22:44:46.013772 21785 consensus_queue.cc:260] T ca8de8e89fbf418bbc2ab79a53b448fe P b711413b9ffb4c8b9072671dd5d5f199 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33257 } } peers { permanent_uuid: "b711413b9ffb4c8b9072671dd5d5f199" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44247 } } peers { permanent_uuid: "af47fa363a39460f9128a46c2a601ec4" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45227 } }
I20260806 22:44:46.013789 21786 raft_consensus.cc:348] T ca8de8e89fbf418bbc2ab79a53b448fe P 92eb5ec5ac944cfca7c5420a72b4652c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33257 } } peers { permanent_uuid: "b711413b9ffb4c8b9072671dd5d5f199" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44247 } } peers { permanent_uuid: "af47fa363a39460f9128a46c2a601ec4" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45227 } }
I20260806 22:44:46.013967 21786 raft_consensus.cc:374] T ca8de8e89fbf418bbc2ab79a53b448fe P 92eb5ec5ac944cfca7c5420a72b4652c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:46.014017 21786 raft_consensus.cc:729] T ca8de8e89fbf418bbc2ab79a53b448fe P 92eb5ec5ac944cfca7c5420a72b4652c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92eb5ec5ac944cfca7c5420a72b4652c, State: Initialized, Role: FOLLOWER
I20260806 22:44:46.014101 21785 ts_tablet_manager.cc:1428] T ca8de8e89fbf418bbc2ab79a53b448fe P b711413b9ffb4c8b9072671dd5d5f199: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260806 22:44:46.014139 21786 consensus_queue.cc:260] T ca8de8e89fbf418bbc2ab79a53b448fe P 92eb5ec5ac944cfca7c5420a72b4652c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33257 } } peers { permanent_uuid: "b711413b9ffb4c8b9072671dd5d5f199" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44247 } } peers { permanent_uuid: "af47fa363a39460f9128a46c2a601ec4" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45227 } }
I20260806 22:44:46.014240 21753 heartbeater.cc:503] Master 127.18.45.254:37773 was elected leader, sending a full tablet report...
I20260806 22:44:46.014441 21786 ts_tablet_manager.cc:1428] T ca8de8e89fbf418bbc2ab79a53b448fe P 92eb5ec5ac944cfca7c5420a72b4652c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:44:46.014326 21787 raft_consensus.cc:348] T ca8de8e89fbf418bbc2ab79a53b448fe P af47fa363a39460f9128a46c2a601ec4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33257 } } peers { permanent_uuid: "b711413b9ffb4c8b9072671dd5d5f199" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44247 } } peers { permanent_uuid: "af47fa363a39460f9128a46c2a601ec4" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45227 } }
I20260806 22:44:46.014488 21787 raft_consensus.cc:374] T ca8de8e89fbf418bbc2ab79a53b448fe P af47fa363a39460f9128a46c2a601ec4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:46.014535 21787 raft_consensus.cc:729] T ca8de8e89fbf418bbc2ab79a53b448fe P af47fa363a39460f9128a46c2a601ec4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af47fa363a39460f9128a46c2a601ec4, State: Initialized, Role: FOLLOWER
I20260806 22:44:46.014602 21354 heartbeater.cc:503] Master 127.18.45.254:37773 was elected leader, sending a full tablet report...
I20260806 22:44:46.014674 21787 consensus_queue.cc:260] T ca8de8e89fbf418bbc2ab79a53b448fe P af47fa363a39460f9128a46c2a601ec4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33257 } } peers { permanent_uuid: "b711413b9ffb4c8b9072671dd5d5f199" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44247 } } peers { permanent_uuid: "af47fa363a39460f9128a46c2a601ec4" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45227 } }
I20260806 22:44:46.014967 21787 ts_tablet_manager.cc:1428] T ca8de8e89fbf418bbc2ab79a53b448fe P af47fa363a39460f9128a46c2a601ec4: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 22:44:46.015138 21554 heartbeater.cc:503] Master 127.18.45.254:37773 was elected leader, sending a full tablet report...
W20260806 22:44:46.065716 21355 tablet.cc:2369] T ca8de8e89fbf418bbc2ab79a53b448fe P 92eb5ec5ac944cfca7c5420a72b4652c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:44:46.203056 21755 tablet.cc:2369] T ca8de8e89fbf418bbc2ab79a53b448fe P b711413b9ffb4c8b9072671dd5d5f199: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:44:46.234457 21796 raft_consensus.cc:482] T ca8de8e89fbf418bbc2ab79a53b448fe P af47fa363a39460f9128a46c2a601ec4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:44:46.234642 21796 raft_consensus.cc:504] T ca8de8e89fbf418bbc2ab79a53b448fe P af47fa363a39460f9128a46c2a601ec4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33257 } } peers { permanent_uuid: "b711413b9ffb4c8b9072671dd5d5f199" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44247 } } peers { permanent_uuid: "af47fa363a39460f9128a46c2a601ec4" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45227 } }
I20260806 22:44:46.235159 21796 leader_election.cc:288] T ca8de8e89fbf418bbc2ab79a53b448fe P af47fa363a39460f9128a46c2a601ec4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 92eb5ec5ac944cfca7c5420a72b4652c (127.18.45.193:33257), b711413b9ffb4c8b9072671dd5d5f199 (127.18.45.195:44247)
I20260806 22:44:46.238329 21781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:46.235545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:33603 (local address 127.18.45.193:33257)
0806 22:44:46.235604 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:46.235606 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:46.235699 (+ 93us) server_negotiation.cc:407] Connection header received
0806 22:44:46.235779 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:46.235780 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:46.235793 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:46.235870 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:44:46.236062 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:46.236551 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:46.237532 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:46.237936 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:46.238027 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:46.238040 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:46.238195 (+ 155us) server_negotiation.cc:299] Negotiation successful
0806 22:44:46.238242 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 22:44:46.238350 21784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:46.235556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:58755 (local address 127.18.45.195:44247)
0806 22:44:46.235617 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:46.235619 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:46.235629 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:44:46.235671 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:46.235671 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:46.235680 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:46.235749 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:44:46.235957 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:46.236501 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:46.237538 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:46.237955 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:46.238044 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:46.238054 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:46.238227 (+ 173us) server_negotiation.cc:299] Negotiation successful
0806 22:44:46.238279 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 22:44:46.238333 21401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:46.235387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.195:44247 (local address 127.18.45.194:58755)
0806 22:44:46.235455 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:46.235470 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:46.235538 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:46.235756 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:46.235758 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:46.235760 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:44:46.235861 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:46.235863 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:46.236598 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:46.236599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:46.237431 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:46.237432 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:46.238040 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:46.238042 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:46.238136 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:46.238139 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:44:46.238206 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 22:44:46.238250 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":25}
I20260806 22:44:46.238348 21797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:46.235409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.193:33257 (local address 127.18.45.194:33603)
0806 22:44:46.235609 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:46.235619 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:46.235716 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:46.235875 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:46.235877 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:46.235878 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:44:46.235968 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:46.235970 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:46.236637 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:46.236638 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:46.237425 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:46.237428 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:46.238019 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:46.238020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:46.238104 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:46.238107 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:44:46.238185 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 22:44:46.238230 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":2,"thread_start_us":102,"threads_started":1}
I20260806 22:44:46.238857 21682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca8de8e89fbf418bbc2ab79a53b448fe" candidate_uuid: "af47fa363a39460f9128a46c2a601ec4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b711413b9ffb4c8b9072671dd5d5f199" is_pre_election: true
I20260806 22:44:46.238948 21290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca8de8e89fbf418bbc2ab79a53b448fe" candidate_uuid: "af47fa363a39460f9128a46c2a601ec4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" is_pre_election: true
I20260806 22:44:46.239152 21290 raft_consensus.cc:2454] T ca8de8e89fbf418bbc2ab79a53b448fe P 92eb5ec5ac944cfca7c5420a72b4652c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af47fa363a39460f9128a46c2a601ec4 in term 0.
I20260806 22:44:46.239154 21682 raft_consensus.cc:2454] T ca8de8e89fbf418bbc2ab79a53b448fe P b711413b9ffb4c8b9072671dd5d5f199 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af47fa363a39460f9128a46c2a601ec4 in term 0.
I20260806 22:44:46.239466 21390 leader_election.cc:302] T ca8de8e89fbf418bbc2ab79a53b448fe P af47fa363a39460f9128a46c2a601ec4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92eb5ec5ac944cfca7c5420a72b4652c, af47fa363a39460f9128a46c2a601ec4; no voters:
I20260806 22:44:46.239686 21796 raft_consensus.cc:2781] T ca8de8e89fbf418bbc2ab79a53b448fe P af47fa363a39460f9128a46c2a601ec4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:44:46.239732 21796 raft_consensus.cc:482] T ca8de8e89fbf418bbc2ab79a53b448fe P af47fa363a39460f9128a46c2a601ec4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:44:46.239748 21796 raft_consensus.cc:3037] T ca8de8e89fbf418bbc2ab79a53b448fe P af47fa363a39460f9128a46c2a601ec4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:46.240630 21796 raft_consensus.cc:504] T ca8de8e89fbf418bbc2ab79a53b448fe P af47fa363a39460f9128a46c2a601ec4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33257 } } peers { permanent_uuid: "b711413b9ffb4c8b9072671dd5d5f199" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44247 } } peers { permanent_uuid: "af47fa363a39460f9128a46c2a601ec4" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45227 } }
I20260806 22:44:46.240796 21796 leader_election.cc:288] T ca8de8e89fbf418bbc2ab79a53b448fe P af47fa363a39460f9128a46c2a601ec4 [CANDIDATE]: Term 1 election: Requested vote from peers 92eb5ec5ac944cfca7c5420a72b4652c (127.18.45.193:33257), b711413b9ffb4c8b9072671dd5d5f199 (127.18.45.195:44247)
I20260806 22:44:46.240964 21290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca8de8e89fbf418bbc2ab79a53b448fe" candidate_uuid: "af47fa363a39460f9128a46c2a601ec4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92eb5ec5ac944cfca7c5420a72b4652c"
I20260806 22:44:46.240996 21682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca8de8e89fbf418bbc2ab79a53b448fe" candidate_uuid: "af47fa363a39460f9128a46c2a601ec4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b711413b9ffb4c8b9072671dd5d5f199"
I20260806 22:44:46.241076 21290 raft_consensus.cc:3037] T ca8de8e89fbf418bbc2ab79a53b448fe P 92eb5ec5ac944cfca7c5420a72b4652c [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:46.241099 21682 raft_consensus.cc:3037] T ca8de8e89fbf418bbc2ab79a53b448fe P b711413b9ffb4c8b9072671dd5d5f199 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:46.241923 21290 raft_consensus.cc:2454] T ca8de8e89fbf418bbc2ab79a53b448fe P 92eb5ec5ac944cfca7c5420a72b4652c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af47fa363a39460f9128a46c2a601ec4 in term 1.
I20260806 22:44:46.241945 21682 raft_consensus.cc:2454] T ca8de8e89fbf418bbc2ab79a53b448fe P b711413b9ffb4c8b9072671dd5d5f199 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af47fa363a39460f9128a46c2a601ec4 in term 1.
I20260806 22:44:46.242102 21390 leader_election.cc:302] T ca8de8e89fbf418bbc2ab79a53b448fe P af47fa363a39460f9128a46c2a601ec4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92eb5ec5ac944cfca7c5420a72b4652c, af47fa363a39460f9128a46c2a601ec4; no voters:
I20260806 22:44:46.242216 21796 raft_consensus.cc:2781] T ca8de8e89fbf418bbc2ab79a53b448fe P af47fa363a39460f9128a46c2a601ec4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:44:46.242473 21796 raft_consensus.cc:686] T ca8de8e89fbf418bbc2ab79a53b448fe P af47fa363a39460f9128a46c2a601ec4 [term 1 LEADER]: Becoming Leader. State: Replica: af47fa363a39460f9128a46c2a601ec4, State: Running, Role: LEADER
I20260806 22:44:46.242589 21796 consensus_queue.cc:237] T ca8de8e89fbf418bbc2ab79a53b448fe P af47fa363a39460f9128a46c2a601ec4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33257 } } peers { permanent_uuid: "b711413b9ffb4c8b9072671dd5d5f199" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44247 } } peers { permanent_uuid: "af47fa363a39460f9128a46c2a601ec4" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45227 } }
I20260806 22:44:46.243741 21068 catalog_manager.cc:5495] T ca8de8e89fbf418bbc2ab79a53b448fe P af47fa363a39460f9128a46c2a601ec4 reported cstate change: term changed from 0 to 1, leader changed from <none> to af47fa363a39460f9128a46c2a601ec4 (127.18.45.194). New cstate: current_term: 1 leader_uuid: "af47fa363a39460f9128a46c2a601ec4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b711413b9ffb4c8b9072671dd5d5f199" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af47fa363a39460f9128a46c2a601ec4" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45227 } health_report { overall_health: HEALTHY } } }
W20260806 22:44:46.263737 21555 tablet.cc:2369] T ca8de8e89fbf418bbc2ab79a53b448fe P af47fa363a39460f9128a46c2a601ec4: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 22:44:46 dist-test-slave-h3ft krb5kdc[21009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056285, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.254@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:44:46.308220 21806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:46.300126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:37773 (local address 127.0.0.1:47982)
0806 22:44:46.300420 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:46.300437 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:46.300562 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:46.300742 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:46.300743 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:46.301193 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:46.301275 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:46.301277 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:46.302016 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:46.302017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:46.302572 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:46.302573 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:46.302964 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:46.302965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:46.303067 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:46.303753 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:44:46.303758 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:46.305414 (+ 1656us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:46.307086 (+ 1672us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:46.307094 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:46.307096 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:46.307337 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:46.307540 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:46.307541 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:46.307541 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:46.307611 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:46.307848 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:46.307855 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:46.307901 (+ 46us) client_negotiation.cc:769] Sending connection context
0806 22:44:46.307957 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 22:44:46.308082 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":128,"threads_started":1}
I20260806 22:44:46.308226 21139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:46.300252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47982 (local address 127.18.45.254:37773)
0806 22:44:46.300383 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:46.300386 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:46.300519 (+ 133us) server_negotiation.cc:407] Connection header received
0806 22:44:46.300648 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:46.300649 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:46.300661 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:46.300737 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:46.301384 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:46.301919 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:46.302659 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:46.302861 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:46.302961 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:46.305515 (+ 2554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:46.305524 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:46.305525 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:46.305532 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:46.306952 (+ 1420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:46.307407 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:46.307408 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:46.307408 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:46.307475 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:46.307669 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:46.307670 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:46.307670 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:46.307791 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:46.307845 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:46.308011 (+ 166us) server_negotiation.cc:299] Negotiation successful
0806 22:44:46.308150 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":82}
I20260806 22:44:46.310801 21068 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:47982:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 22:44:46.311125 21068 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:44:46.313015 21640 tablet_service.cc:1467] Processing CreateTablet for tablet b3676c4be8794f278d8ef78465e765fa (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=df68e2d6b6084e1f892f9152e44fba81]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:44:46.313028 21464 tablet_service.cc:1467] Processing CreateTablet for tablet b3676c4be8794f278d8ef78465e765fa (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=df68e2d6b6084e1f892f9152e44fba81]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:44:46.313014 21260 tablet_service.cc:1467] Processing CreateTablet for tablet b3676c4be8794f278d8ef78465e765fa (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=df68e2d6b6084e1f892f9152e44fba81]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:44:46.313249 21640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3676c4be8794f278d8ef78465e765fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:46.313249 21464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3676c4be8794f278d8ef78465e765fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:46.313251 21260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3676c4be8794f278d8ef78465e765fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:46.314802 21785 tablet_bootstrap.cc:492] T b3676c4be8794f278d8ef78465e765fa P b711413b9ffb4c8b9072671dd5d5f199: Bootstrap starting.
I20260806 22:44:46.315171 21787 tablet_bootstrap.cc:492] T b3676c4be8794f278d8ef78465e765fa P af47fa363a39460f9128a46c2a601ec4: Bootstrap starting.
I20260806 22:44:46.315650 21785 tablet_bootstrap.cc:654] T b3676c4be8794f278d8ef78465e765fa P b711413b9ffb4c8b9072671dd5d5f199: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:46.315652 21787 tablet_bootstrap.cc:654] T b3676c4be8794f278d8ef78465e765fa P af47fa363a39460f9128a46c2a601ec4: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:46.316466 21786 tablet_bootstrap.cc:492] T b3676c4be8794f278d8ef78465e765fa P 92eb5ec5ac944cfca7c5420a72b4652c: Bootstrap starting.
I20260806 22:44:46.316533 21785 tablet_bootstrap.cc:492] T b3676c4be8794f278d8ef78465e765fa P b711413b9ffb4c8b9072671dd5d5f199: No bootstrap required, opened a new log
I20260806 22:44:46.316594 21785 ts_tablet_manager.cc:1397] T b3676c4be8794f278d8ef78465e765fa P b711413b9ffb4c8b9072671dd5d5f199: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 22:44:46.316768 21787 tablet_bootstrap.cc:492] T b3676c4be8794f278d8ef78465e765fa P af47fa363a39460f9128a46c2a601ec4: No bootstrap required, opened a new log
I20260806 22:44:46.316828 21787 ts_tablet_manager.cc:1397] T b3676c4be8794f278d8ef78465e765fa P af47fa363a39460f9128a46c2a601ec4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:44:46.316802 21785 raft_consensus.cc:348] T b3676c4be8794f278d8ef78465e765fa P b711413b9ffb4c8b9072671dd5d5f199 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33257 } } peers { permanent_uuid: "af47fa363a39460f9128a46c2a601ec4" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45227 } } peers { permanent_uuid: "b711413b9ffb4c8b9072671dd5d5f199" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44247 } }
I20260806 22:44:46.316912 21785 raft_consensus.cc:374] T b3676c4be8794f278d8ef78465e765fa P b711413b9ffb4c8b9072671dd5d5f199 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:46.316962 21785 raft_consensus.cc:729] T b3676c4be8794f278d8ef78465e765fa P b711413b9ffb4c8b9072671dd5d5f199 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b711413b9ffb4c8b9072671dd5d5f199, State: Initialized, Role: FOLLOWER
I20260806 22:44:46.316969 21786 tablet_bootstrap.cc:654] T b3676c4be8794f278d8ef78465e765fa P 92eb5ec5ac944cfca7c5420a72b4652c: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:46.317019 21785 consensus_queue.cc:260] T b3676c4be8794f278d8ef78465e765fa P b711413b9ffb4c8b9072671dd5d5f199 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33257 } } peers { permanent_uuid: "af47fa363a39460f9128a46c2a601ec4" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45227 } } peers { permanent_uuid: "b711413b9ffb4c8b9072671dd5d5f199" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44247 } }
I20260806 22:44:46.317003 21787 raft_consensus.cc:348] T b3676c4be8794f278d8ef78465e765fa P af47fa363a39460f9128a46c2a601ec4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33257 } } peers { permanent_uuid: "af47fa363a39460f9128a46c2a601ec4" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45227 } } peers { permanent_uuid: "b711413b9ffb4c8b9072671dd5d5f199" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44247 } }
I20260806 22:44:46.317072 21787 raft_consensus.cc:374] T b3676c4be8794f278d8ef78465e765fa P af47fa363a39460f9128a46c2a601ec4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:46.317094 21787 raft_consensus.cc:729] T b3676c4be8794f278d8ef78465e765fa P af47fa363a39460f9128a46c2a601ec4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af47fa363a39460f9128a46c2a601ec4, State: Initialized, Role: FOLLOWER
I20260806 22:44:46.317126 21785 ts_tablet_manager.cc:1428] T b3676c4be8794f278d8ef78465e765fa P b711413b9ffb4c8b9072671dd5d5f199: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260806 22:44:46.317143 21787 consensus_queue.cc:260] T b3676c4be8794f278d8ef78465e765fa P af47fa363a39460f9128a46c2a601ec4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33257 } } peers { permanent_uuid: "af47fa363a39460f9128a46c2a601ec4" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45227 } } peers { permanent_uuid: "b711413b9ffb4c8b9072671dd5d5f199" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44247 } }
I20260806 22:44:46.317253 21787 ts_tablet_manager.cc:1428] T b3676c4be8794f278d8ef78465e765fa P af47fa363a39460f9128a46c2a601ec4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:46.317309 21793 tablet_replica.cc:439] T b3676c4be8794f278d8ef78465e765fa P b711413b9ffb4c8b9072671dd5d5f199: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33257 } } peers { permanent_uuid: "af47fa363a39460f9128a46c2a601ec4" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45227 } } peers { permanent_uuid: "b711413b9ffb4c8b9072671dd5d5f199" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44247 } } }
I20260806 22:44:46.317395 21793 tablet_replica.cc:442] T b3676c4be8794f278d8ef78465e765fa P b711413b9ffb4c8b9072671dd5d5f199: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:44:46.317761 21796 tablet_replica.cc:439] T b3676c4be8794f278d8ef78465e765fa P af47fa363a39460f9128a46c2a601ec4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33257 } } peers { permanent_uuid: "af47fa363a39460f9128a46c2a601ec4" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45227 } } peers { permanent_uuid: "b711413b9ffb4c8b9072671dd5d5f199" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44247 } } }
I20260806 22:44:46.317867 21796 tablet_replica.cc:442] T b3676c4be8794f278d8ef78465e765fa P af47fa363a39460f9128a46c2a601ec4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:44:46.318454 21786 tablet_bootstrap.cc:492] T b3676c4be8794f278d8ef78465e765fa P 92eb5ec5ac944cfca7c5420a72b4652c: No bootstrap required, opened a new log
I20260806 22:44:46.318518 21786 ts_tablet_manager.cc:1397] T b3676c4be8794f278d8ef78465e765fa P 92eb5ec5ac944cfca7c5420a72b4652c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:44:46.318666 21786 raft_consensus.cc:348] T b3676c4be8794f278d8ef78465e765fa P 92eb5ec5ac944cfca7c5420a72b4652c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33257 } } peers { permanent_uuid: "af47fa363a39460f9128a46c2a601ec4" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45227 } } peers { permanent_uuid: "b711413b9ffb4c8b9072671dd5d5f199" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44247 } }
I20260806 22:44:46.318723 21786 raft_consensus.cc:374] T b3676c4be8794f278d8ef78465e765fa P 92eb5ec5ac944cfca7c5420a72b4652c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:46.318737 21786 raft_consensus.cc:729] T b3676c4be8794f278d8ef78465e765fa P 92eb5ec5ac944cfca7c5420a72b4652c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92eb5ec5ac944cfca7c5420a72b4652c, State: Initialized, Role: FOLLOWER
I20260806 22:44:46.318800 21786 consensus_queue.cc:260] T b3676c4be8794f278d8ef78465e765fa P 92eb5ec5ac944cfca7c5420a72b4652c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33257 } } peers { permanent_uuid: "af47fa363a39460f9128a46c2a601ec4" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45227 } } peers { permanent_uuid: "b711413b9ffb4c8b9072671dd5d5f199" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44247 } }
I20260806 22:44:46.318907 21786 ts_tablet_manager.cc:1428] T b3676c4be8794f278d8ef78465e765fa P 92eb5ec5ac944cfca7c5420a72b4652c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:46.319046 21795 tablet_replica.cc:439] T b3676c4be8794f278d8ef78465e765fa P 92eb5ec5ac944cfca7c5420a72b4652c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33257 } } peers { permanent_uuid: "af47fa363a39460f9128a46c2a601ec4" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45227 } } peers { permanent_uuid: "b711413b9ffb4c8b9072671dd5d5f199" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44247 } } }
I20260806 22:44:46.319164 21795 tablet_replica.cc:442] T b3676c4be8794f278d8ef78465e765fa P 92eb5ec5ac944cfca7c5420a72b4652c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:44:46.439307 21796 raft_consensus.cc:482] T b3676c4be8794f278d8ef78465e765fa P af47fa363a39460f9128a46c2a601ec4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:44:46.439450 21796 raft_consensus.cc:504] T b3676c4be8794f278d8ef78465e765fa P af47fa363a39460f9128a46c2a601ec4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33257 } } peers { permanent_uuid: "af47fa363a39460f9128a46c2a601ec4" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45227 } } peers { permanent_uuid: "b711413b9ffb4c8b9072671dd5d5f199" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44247 } }
I20260806 22:44:46.439698 21796 leader_election.cc:288] T b3676c4be8794f278d8ef78465e765fa P af47fa363a39460f9128a46c2a601ec4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 92eb5ec5ac944cfca7c5420a72b4652c (127.18.45.193:33257), b711413b9ffb4c8b9072671dd5d5f199 (127.18.45.195:44247)
I20260806 22:44:46.440052 21290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3676c4be8794f278d8ef78465e765fa" candidate_uuid: "af47fa363a39460f9128a46c2a601ec4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" is_pre_election: true
I20260806 22:44:46.440043 21682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3676c4be8794f278d8ef78465e765fa" candidate_uuid: "af47fa363a39460f9128a46c2a601ec4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b711413b9ffb4c8b9072671dd5d5f199" is_pre_election: true
I20260806 22:44:46.440168 21290 raft_consensus.cc:2454] T b3676c4be8794f278d8ef78465e765fa P 92eb5ec5ac944cfca7c5420a72b4652c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af47fa363a39460f9128a46c2a601ec4 in term 0.
I20260806 22:44:46.440198 21682 raft_consensus.cc:2454] T b3676c4be8794f278d8ef78465e765fa P b711413b9ffb4c8b9072671dd5d5f199 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af47fa363a39460f9128a46c2a601ec4 in term 0.
I20260806 22:44:46.440353 21390 leader_election.cc:302] T b3676c4be8794f278d8ef78465e765fa P af47fa363a39460f9128a46c2a601ec4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92eb5ec5ac944cfca7c5420a72b4652c, af47fa363a39460f9128a46c2a601ec4; no voters:
I20260806 22:44:46.440477 21796 raft_consensus.cc:2781] T b3676c4be8794f278d8ef78465e765fa P af47fa363a39460f9128a46c2a601ec4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:44:46.440541 21796 raft_consensus.cc:482] T b3676c4be8794f278d8ef78465e765fa P af47fa363a39460f9128a46c2a601ec4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:44:46.440585 21796 raft_consensus.cc:3037] T b3676c4be8794f278d8ef78465e765fa P af47fa363a39460f9128a46c2a601ec4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:46.441545 21796 raft_consensus.cc:504] T b3676c4be8794f278d8ef78465e765fa P af47fa363a39460f9128a46c2a601ec4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33257 } } peers { permanent_uuid: "af47fa363a39460f9128a46c2a601ec4" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45227 } } peers { permanent_uuid: "b711413b9ffb4c8b9072671dd5d5f199" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44247 } }
I20260806 22:44:46.441712 21796 leader_election.cc:288] T b3676c4be8794f278d8ef78465e765fa P af47fa363a39460f9128a46c2a601ec4 [CANDIDATE]: Term 1 election: Requested vote from peers 92eb5ec5ac944cfca7c5420a72b4652c (127.18.45.193:33257), b711413b9ffb4c8b9072671dd5d5f199 (127.18.45.195:44247)
I20260806 22:44:46.441941 21290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3676c4be8794f278d8ef78465e765fa" candidate_uuid: "af47fa363a39460f9128a46c2a601ec4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92eb5ec5ac944cfca7c5420a72b4652c"
I20260806 22:44:46.441931 21682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3676c4be8794f278d8ef78465e765fa" candidate_uuid: "af47fa363a39460f9128a46c2a601ec4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b711413b9ffb4c8b9072671dd5d5f199"
I20260806 22:44:46.442031 21290 raft_consensus.cc:3037] T b3676c4be8794f278d8ef78465e765fa P 92eb5ec5ac944cfca7c5420a72b4652c [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:46.442050 21682 raft_consensus.cc:3037] T b3676c4be8794f278d8ef78465e765fa P b711413b9ffb4c8b9072671dd5d5f199 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:46.442793 21290 raft_consensus.cc:2454] T b3676c4be8794f278d8ef78465e765fa P 92eb5ec5ac944cfca7c5420a72b4652c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af47fa363a39460f9128a46c2a601ec4 in term 1.
I20260806 22:44:46.442876 21682 raft_consensus.cc:2454] T b3676c4be8794f278d8ef78465e765fa P b711413b9ffb4c8b9072671dd5d5f199 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af47fa363a39460f9128a46c2a601ec4 in term 1.
I20260806 22:44:46.442976 21390 leader_election.cc:302] T b3676c4be8794f278d8ef78465e765fa P af47fa363a39460f9128a46c2a601ec4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92eb5ec5ac944cfca7c5420a72b4652c, af47fa363a39460f9128a46c2a601ec4; no voters:
I20260806 22:44:46.443106 21796 raft_consensus.cc:2781] T b3676c4be8794f278d8ef78465e765fa P af47fa363a39460f9128a46c2a601ec4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:44:46.443166 21796 raft_consensus.cc:686] T b3676c4be8794f278d8ef78465e765fa P af47fa363a39460f9128a46c2a601ec4 [term 1 LEADER]: Becoming Leader. State: Replica: af47fa363a39460f9128a46c2a601ec4, State: Running, Role: LEADER
I20260806 22:44:46.443212 21796 consensus_queue.cc:237] T b3676c4be8794f278d8ef78465e765fa P af47fa363a39460f9128a46c2a601ec4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33257 } } peers { permanent_uuid: "af47fa363a39460f9128a46c2a601ec4" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45227 } } peers { permanent_uuid: "b711413b9ffb4c8b9072671dd5d5f199" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44247 } }
I20260806 22:44:46.443626 21798 tablet_replica.cc:439] T b3676c4be8794f278d8ef78465e765fa P af47fa363a39460f9128a46c2a601ec4: TxnStatusTablet state changed. Reason: New leader af47fa363a39460f9128a46c2a601ec4. Latest consensus state: current_term: 1 leader_uuid: "af47fa363a39460f9128a46c2a601ec4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33257 } } peers { permanent_uuid: "af47fa363a39460f9128a46c2a601ec4" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45227 } } peers { permanent_uuid: "b711413b9ffb4c8b9072671dd5d5f199" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44247 } } }
I20260806 22:44:46.443728 21798 tablet_replica.cc:442] T b3676c4be8794f278d8ef78465e765fa P af47fa363a39460f9128a46c2a601ec4: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:44:46.444036 21813 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 22:44:46.444036 21068 catalog_manager.cc:5495] T b3676c4be8794f278d8ef78465e765fa P af47fa363a39460f9128a46c2a601ec4 reported cstate change: term changed from 0 to 1, leader changed from <none> to af47fa363a39460f9128a46c2a601ec4 (127.18.45.194). New cstate: current_term: 1 leader_uuid: "af47fa363a39460f9128a46c2a601ec4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af47fa363a39460f9128a46c2a601ec4" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b711413b9ffb4c8b9072671dd5d5f199" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44247 } health_report { overall_health: UNKNOWN } } }
I20260806 22:44:46.444232 21813 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 22:44:46.467196 21806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:46.463814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.194:45227 (local address 127.0.0.1:40494)
0806 22:44:46.463933 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:46.463948 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:46.464034 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:46.464295 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:46.464296 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:46.464298 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:44:46.464386 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:46.464389 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:46.465374 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:46.465375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:46.465998 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:46.465999 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:46.466505 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:46.466505 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:46.466587 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:46.466606 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:44:46.466950 (+ 344us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:44:46.466958 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 22:44:46.467072 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 22:44:46.467128 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260806 22:44:46.467178 21783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:46.463905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40494 (local address 127.18.45.194:45227)
0806 22:44:46.463986 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:46.463988 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:46.464008 (+ 20us) server_negotiation.cc:407] Connection header received
0806 22:44:46.464166 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:46.464168 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:46.464198 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:46.464276 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:44:46.464464 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:46.465283 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:46.466091 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:46.466425 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:46.466509 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:46.466712 (+ 203us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:44:46.466866 (+ 154us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:44:46.466960 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:46.467035 (+ 75us) server_negotiation.cc:299] Negotiation successful
0806 22:44:46.467065 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 22:44:46.469754 21290 raft_consensus.cc:1261] T b3676c4be8794f278d8ef78465e765fa P 92eb5ec5ac944cfca7c5420a72b4652c [term 1 FOLLOWER]: Refusing update from remote peer af47fa363a39460f9128a46c2a601ec4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:44:46.469754 21682 raft_consensus.cc:1261] T b3676c4be8794f278d8ef78465e765fa P b711413b9ffb4c8b9072671dd5d5f199 [term 1 FOLLOWER]: Refusing update from remote peer af47fa363a39460f9128a46c2a601ec4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:44:46.470264 21798 consensus_queue.cc:1035] T b3676c4be8794f278d8ef78465e765fa P af47fa363a39460f9128a46c2a601ec4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b711413b9ffb4c8b9072671dd5d5f199" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:44:46.470520 21796 consensus_queue.cc:1035] T b3676c4be8794f278d8ef78465e765fa P af47fa363a39460f9128a46c2a601ec4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:44:46.471887 21795 tablet_replica.cc:439] T b3676c4be8794f278d8ef78465e765fa P 92eb5ec5ac944cfca7c5420a72b4652c: TxnStatusTablet state changed. Reason: New leader af47fa363a39460f9128a46c2a601ec4. Latest consensus state: current_term: 1 leader_uuid: "af47fa363a39460f9128a46c2a601ec4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33257 } } peers { permanent_uuid: "af47fa363a39460f9128a46c2a601ec4" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45227 } } peers { permanent_uuid: "b711413b9ffb4c8b9072671dd5d5f199" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44247 } } }
I20260806 22:44:46.471994 21795 tablet_replica.cc:442] T b3676c4be8794f278d8ef78465e765fa P 92eb5ec5ac944cfca7c5420a72b4652c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:44:46.472194 21793 tablet_replica.cc:439] T b3676c4be8794f278d8ef78465e765fa P b711413b9ffb4c8b9072671dd5d5f199: TxnStatusTablet state changed. Reason: New leader af47fa363a39460f9128a46c2a601ec4. Latest consensus state: current_term: 1 leader_uuid: "af47fa363a39460f9128a46c2a601ec4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33257 } } peers { permanent_uuid: "af47fa363a39460f9128a46c2a601ec4" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45227 } } peers { permanent_uuid: "b711413b9ffb4c8b9072671dd5d5f199" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44247 } } }
I20260806 22:44:46.472317 21793 tablet_replica.cc:442] T b3676c4be8794f278d8ef78465e765fa P b711413b9ffb4c8b9072671dd5d5f199: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:44:46.473285 21796 tablet_replica.cc:439] T b3676c4be8794f278d8ef78465e765fa P af47fa363a39460f9128a46c2a601ec4: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "af47fa363a39460f9128a46c2a601ec4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33257 } } peers { permanent_uuid: "af47fa363a39460f9128a46c2a601ec4" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45227 } } peers { permanent_uuid: "b711413b9ffb4c8b9072671dd5d5f199" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44247 } } }
I20260806 22:44:46.473388 21796 tablet_replica.cc:442] T b3676c4be8794f278d8ef78465e765fa P af47fa363a39460f9128a46c2a601ec4: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:44:46.474110 21795 tablet_replica.cc:439] T b3676c4be8794f278d8ef78465e765fa P 92eb5ec5ac944cfca7c5420a72b4652c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "af47fa363a39460f9128a46c2a601ec4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33257 } } peers { permanent_uuid: "af47fa363a39460f9128a46c2a601ec4" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45227 } } peers { permanent_uuid: "b711413b9ffb4c8b9072671dd5d5f199" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44247 } } }
I20260806 22:44:46.474200 21795 tablet_replica.cc:442] T b3676c4be8794f278d8ef78465e765fa P 92eb5ec5ac944cfca7c5420a72b4652c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:44:46.474771 21819 mvcc.cc:205] Tried to move back new op lower bound from 7315686549376311296 to 7315686549271945216. Current Snapshot: MvccSnapshot[applied={T|T < 7315686549376311296 or (T in {7315686549376311296})}]
I20260806 22:44:46.475630 21798 tablet_replica.cc:439] T b3676c4be8794f278d8ef78465e765fa P af47fa363a39460f9128a46c2a601ec4: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "af47fa363a39460f9128a46c2a601ec4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33257 } } peers { permanent_uuid: "af47fa363a39460f9128a46c2a601ec4" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45227 } } peers { permanent_uuid: "b711413b9ffb4c8b9072671dd5d5f199" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44247 } } }
I20260806 22:44:46.475716 21798 tablet_replica.cc:442] T b3676c4be8794f278d8ef78465e765fa P af47fa363a39460f9128a46c2a601ec4: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:44:46.476471 21793 tablet_replica.cc:439] T b3676c4be8794f278d8ef78465e765fa P b711413b9ffb4c8b9072671dd5d5f199: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "af47fa363a39460f9128a46c2a601ec4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33257 } } peers { permanent_uuid: "af47fa363a39460f9128a46c2a601ec4" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45227 } } peers { permanent_uuid: "b711413b9ffb4c8b9072671dd5d5f199" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44247 } } }
I20260806 22:44:46.476595 21793 tablet_replica.cc:442] T b3676c4be8794f278d8ef78465e765fa P b711413b9ffb4c8b9072671dd5d5f199: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:44:46.476722 21818 mvcc.cc:205] Tried to move back new op lower bound from 7315686549376311296 to 7315686549271945216. Current Snapshot: MvccSnapshot[applied={T|T < 7315686549376311296}]
I20260806 22:44:46.478786 21783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:46.475957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40510 (local address 127.18.45.194:45227)
0806 22:44:46.476022 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:46.476024 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:46.476033 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:44:46.476057 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:46.476058 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:46.476079 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:46.476140 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:44:46.476379 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:46.477167 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:46.477980 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:46.478298 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:46.478372 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:46.478535 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:44:46.478579 (+ 44us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:44:46.478639 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:46.478681 (+ 42us) server_negotiation.cc:299] Negotiation successful
0806 22:44:46.478704 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:44:46.485029 21783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:46.481938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:52951 (local address 127.18.45.194:45227)
0806 22:44:46.481985 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:46.481988 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:46.482149 (+ 161us) server_negotiation.cc:407] Connection header received
0806 22:44:46.482215 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:46.482216 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:46.482224 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:46.482270 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:44:46.482464 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:46.482943 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:46.484142 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:46.484338 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:46.484406 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:46.484700 (+ 294us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:44:46.484743 (+ 43us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:44:46.484799 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:46.484882 (+ 83us) server_negotiation.cc:299] Negotiation successful
0806 22:44:46.484923 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":15,"server-negotiator.queue_time_us":20}
I20260806 22:44:46.485210 21829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:46.481880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.194:45227 (local address 127.18.45.194:52951)
0806 22:44:46.482072 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:46.482085 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:46.482159 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:46.482291 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:46.482292 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:46.482294 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:44:46.482374 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:46.482376 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:46.483054 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:46.483055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:46.484030 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:46.484032 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:46.484450 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:46.484452 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:46.484545 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:46.484563 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:44:46.484821 (+ 258us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:44:46.484823 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:44:46.484879 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 22:44:46.484953 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":2,"thread_start_us":97,"threads_started":1}
I20260806 22:44:46.488585 21290 raft_consensus.cc:1261] T ca8de8e89fbf418bbc2ab79a53b448fe P 92eb5ec5ac944cfca7c5420a72b4652c [term 1 FOLLOWER]: Refusing update from remote peer af47fa363a39460f9128a46c2a601ec4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:44:46.488584 21682 raft_consensus.cc:1261] T ca8de8e89fbf418bbc2ab79a53b448fe P b711413b9ffb4c8b9072671dd5d5f199 [term 1 FOLLOWER]: Refusing update from remote peer af47fa363a39460f9128a46c2a601ec4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:44:46.488821 21830 consensus_queue.cc:1035] T ca8de8e89fbf418bbc2ab79a53b448fe P af47fa363a39460f9128a46c2a601ec4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b711413b9ffb4c8b9072671dd5d5f199" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:44:46.488936 21796 consensus_queue.cc:1035] T ca8de8e89fbf418bbc2ab79a53b448fe P af47fa363a39460f9128a46c2a601ec4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:44:46.491339 21817 mvcc.cc:205] Tried to move back new op lower bound from 7315686549455720448 to 7315686548450578432. Current Snapshot: MvccSnapshot[applied={T|T < 7315686549455720448}]
I20260806 22:44:46.506254 21066 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47968:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:44:46.506428 21066 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47968:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:44:46.507395 21066 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e54ba47edd124325be88cc0cc4df70b5: Sending DeleteTablet for 3 replicas of tablet ca8de8e89fbf418bbc2ab79a53b448fe
I20260806 22:44:46.507788 21640 tablet_service.cc:1514] Processing DeleteTablet for tablet ca8de8e89fbf418bbc2ab79a53b448fe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:44:46 UTC) from {username='kudu', principal='kudu/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:46594
I20260806 22:44:46.507812 21260 tablet_service.cc:1514] Processing DeleteTablet for tablet ca8de8e89fbf418bbc2ab79a53b448fe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:44:46 UTC) from {username='kudu', principal='kudu/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:47084
I20260806 22:44:46.508205 21464 tablet_service.cc:1514] Processing DeleteTablet for tablet ca8de8e89fbf418bbc2ab79a53b448fe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:44:46 UTC) from {username='kudu', principal='kudu/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:40484
I20260806 22:44:46.508548 21838 tablet_replica.cc:330] T ca8de8e89fbf418bbc2ab79a53b448fe P af47fa363a39460f9128a46c2a601ec4: stopping tablet replica
I20260806 22:44:46.508489 21066 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47968:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:44:46.508663 21066 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:44:46.508718 21838 raft_consensus.cc:2229] T ca8de8e89fbf418bbc2ab79a53b448fe P af47fa363a39460f9128a46c2a601ec4 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:44:46.508862 21838 raft_consensus.cc:2258] T ca8de8e89fbf418bbc2ab79a53b448fe P af47fa363a39460f9128a46c2a601ec4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:44:46.509435 21838 ts_tablet_manager.cc:1905] T ca8de8e89fbf418bbc2ab79a53b448fe P af47fa363a39460f9128a46c2a601ec4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:44:46.510134 21640 tablet_service.cc:1467] Processing CreateTablet for tablet 8522531482fc4eba8ad03aad8ba6c201 (DEFAULT_TABLE table=test-table [id=54b3a18a7f0340ab899c669a049514f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:44:46.510212 21260 tablet_service.cc:1467] Processing CreateTablet for tablet 8522531482fc4eba8ad03aad8ba6c201 (DEFAULT_TABLE table=test-table [id=54b3a18a7f0340ab899c669a049514f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:44:46.510291 21640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8522531482fc4eba8ad03aad8ba6c201. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:46.510370 21260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8522531482fc4eba8ad03aad8ba6c201. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:46.511109 21464 tablet_service.cc:1467] Processing CreateTablet for tablet 8522531482fc4eba8ad03aad8ba6c201 (DEFAULT_TABLE table=test-table [id=54b3a18a7f0340ab899c669a049514f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:44:46.511224 21464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8522531482fc4eba8ad03aad8ba6c201. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:46.511611 21838 ts_tablet_manager.cc:1918] T ca8de8e89fbf418bbc2ab79a53b448fe P af47fa363a39460f9128a46c2a601ec4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:44:46.511683 21838 log.cc:1196] T ca8de8e89fbf418bbc2ab79a53b448fe P af47fa363a39460f9128a46c2a601ec4: Deleting WAL directory at /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/ts-1/wal/wals/ca8de8e89fbf418bbc2ab79a53b448fe
I20260806 22:44:46.511984 21838 ts_tablet_manager.cc:1939] T ca8de8e89fbf418bbc2ab79a53b448fe P af47fa363a39460f9128a46c2a601ec4: Deleting consensus metadata
I20260806 22:44:46.512252 21785 tablet_bootstrap.cc:492] T 8522531482fc4eba8ad03aad8ba6c201 P b711413b9ffb4c8b9072671dd5d5f199: Bootstrap starting.
I20260806 22:44:46.512535 21787 tablet_bootstrap.cc:492] T 8522531482fc4eba8ad03aad8ba6c201 P af47fa363a39460f9128a46c2a601ec4: Bootstrap starting.
I20260806 22:44:46.512778 21785 tablet_bootstrap.cc:654] T 8522531482fc4eba8ad03aad8ba6c201 P b711413b9ffb4c8b9072671dd5d5f199: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:46.513063 21787 tablet_bootstrap.cc:654] T 8522531482fc4eba8ad03aad8ba6c201 P af47fa363a39460f9128a46c2a601ec4: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:46.513437 21785 tablet_bootstrap.cc:492] T 8522531482fc4eba8ad03aad8ba6c201 P b711413b9ffb4c8b9072671dd5d5f199: No bootstrap required, opened a new log
I20260806 22:44:46.513484 21785 ts_tablet_manager.cc:1397] T 8522531482fc4eba8ad03aad8ba6c201 P b711413b9ffb4c8b9072671dd5d5f199: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 22:44:46.513540 21050 catalog_manager.cc:4841] TS af47fa363a39460f9128a46c2a601ec4 (127.18.45.194:45227): tablet ca8de8e89fbf418bbc2ab79a53b448fe (table test-table [id=a60c867d5f064f9493512807af072e15]) successfully deleted
I20260806 22:44:46.513652 21785 raft_consensus.cc:348] T 8522531482fc4eba8ad03aad8ba6c201 P b711413b9ffb4c8b9072671dd5d5f199 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33257 } } peers { permanent_uuid: "af47fa363a39460f9128a46c2a601ec4" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45227 } } peers { permanent_uuid: "b711413b9ffb4c8b9072671dd5d5f199" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44247 } }
I20260806 22:44:46.513720 21785 raft_consensus.cc:374] T 8522531482fc4eba8ad03aad8ba6c201 P b711413b9ffb4c8b9072671dd5d5f199 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:46.513737 21785 raft_consensus.cc:729] T 8522531482fc4eba8ad03aad8ba6c201 P b711413b9ffb4c8b9072671dd5d5f199 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b711413b9ffb4c8b9072671dd5d5f199, State: Initialized, Role: FOLLOWER
I20260806 22:44:46.513778 21785 consensus_queue.cc:260] T 8522531482fc4eba8ad03aad8ba6c201 P b711413b9ffb4c8b9072671dd5d5f199 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33257 } } peers { permanent_uuid: "af47fa363a39460f9128a46c2a601ec4" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45227 } } peers { permanent_uuid: "b711413b9ffb4c8b9072671dd5d5f199" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44247 } }
I20260806 22:44:46.513863 21785 ts_tablet_manager.cc:1428] T 8522531482fc4eba8ad03aad8ba6c201 P b711413b9ffb4c8b9072671dd5d5f199: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:46.515090 21786 tablet_bootstrap.cc:492] T 8522531482fc4eba8ad03aad8ba6c201 P 92eb5ec5ac944cfca7c5420a72b4652c: Bootstrap starting.
I20260806 22:44:46.515717 21787 tablet_bootstrap.cc:492] T 8522531482fc4eba8ad03aad8ba6c201 P af47fa363a39460f9128a46c2a601ec4: No bootstrap required, opened a new log
I20260806 22:44:46.515765 21787 ts_tablet_manager.cc:1397] T 8522531482fc4eba8ad03aad8ba6c201 P af47fa363a39460f9128a46c2a601ec4: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 22:44:46.515882 21787 raft_consensus.cc:348] T 8522531482fc4eba8ad03aad8ba6c201 P af47fa363a39460f9128a46c2a601ec4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33257 } } peers { permanent_uuid: "af47fa363a39460f9128a46c2a601ec4" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45227 } } peers { permanent_uuid: "b711413b9ffb4c8b9072671dd5d5f199" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44247 } }
I20260806 22:44:46.515947 21787 raft_consensus.cc:374] T 8522531482fc4eba8ad03aad8ba6c201 P af47fa363a39460f9128a46c2a601ec4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:46.515967 21787 raft_consensus.cc:729] T 8522531482fc4eba8ad03aad8ba6c201 P af47fa363a39460f9128a46c2a601ec4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af47fa363a39460f9128a46c2a601ec4, State: Initialized, Role: FOLLOWER
I20260806 22:44:46.515985 21786 tablet_bootstrap.cc:654] T 8522531482fc4eba8ad03aad8ba6c201 P 92eb5ec5ac944cfca7c5420a72b4652c: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:46.516022 21787 consensus_queue.cc:260] T 8522531482fc4eba8ad03aad8ba6c201 P af47fa363a39460f9128a46c2a601ec4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33257 } } peers { permanent_uuid: "af47fa363a39460f9128a46c2a601ec4" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45227 } } peers { permanent_uuid: "b711413b9ffb4c8b9072671dd5d5f199" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44247 } }
I20260806 22:44:46.516119 21787 ts_tablet_manager.cc:1428] T 8522531482fc4eba8ad03aad8ba6c201 P af47fa363a39460f9128a46c2a601ec4: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 22:44:46.516827 21786 tablet_bootstrap.cc:492] T 8522531482fc4eba8ad03aad8ba6c201 P 92eb5ec5ac944cfca7c5420a72b4652c: No bootstrap required, opened a new log
I20260806 22:44:46.516891 21786 ts_tablet_manager.cc:1397] T 8522531482fc4eba8ad03aad8ba6c201 P 92eb5ec5ac944cfca7c5420a72b4652c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:44:46.517087 21786 raft_consensus.cc:348] T 8522531482fc4eba8ad03aad8ba6c201 P 92eb5ec5ac944cfca7c5420a72b4652c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33257 } } peers { permanent_uuid: "af47fa363a39460f9128a46c2a601ec4" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45227 } } peers { permanent_uuid: "b711413b9ffb4c8b9072671dd5d5f199" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44247 } }
I20260806 22:44:46.517176 21786 raft_consensus.cc:374] T 8522531482fc4eba8ad03aad8ba6c201 P 92eb5ec5ac944cfca7c5420a72b4652c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:46.517196 21786 raft_consensus.cc:729] T 8522531482fc4eba8ad03aad8ba6c201 P 92eb5ec5ac944cfca7c5420a72b4652c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92eb5ec5ac944cfca7c5420a72b4652c, State: Initialized, Role: FOLLOWER
I20260806 22:44:46.517242 21786 consensus_queue.cc:260] T 8522531482fc4eba8ad03aad8ba6c201 P 92eb5ec5ac944cfca7c5420a72b4652c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33257 } } peers { permanent_uuid: "af47fa363a39460f9128a46c2a601ec4" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45227 } } peers { permanent_uuid: "b711413b9ffb4c8b9072671dd5d5f199" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44247 } }
I20260806 22:44:46.517338 21786 ts_tablet_manager.cc:1428] T 8522531482fc4eba8ad03aad8ba6c201 P 92eb5ec5ac944cfca7c5420a72b4652c: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 22:44:46.519141 21836 tablet_replica.cc:330] T ca8de8e89fbf418bbc2ab79a53b448fe P b711413b9ffb4c8b9072671dd5d5f199: stopping tablet replica
I20260806 22:44:46.519191 21837 tablet_replica.cc:330] T ca8de8e89fbf418bbc2ab79a53b448fe P 92eb5ec5ac944cfca7c5420a72b4652c: stopping tablet replica
I20260806 22:44:46.519347 21836 raft_consensus.cc:2229] T ca8de8e89fbf418bbc2ab79a53b448fe P b711413b9ffb4c8b9072671dd5d5f199 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:44:46.519356 21837 raft_consensus.cc:2229] T ca8de8e89fbf418bbc2ab79a53b448fe P 92eb5ec5ac944cfca7c5420a72b4652c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:44:46.519498 21837 raft_consensus.cc:2258] T ca8de8e89fbf418bbc2ab79a53b448fe P 92eb5ec5ac944cfca7c5420a72b4652c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:44:46.519505 21836 raft_consensus.cc:2258] T ca8de8e89fbf418bbc2ab79a53b448fe P b711413b9ffb4c8b9072671dd5d5f199 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:44:46.519959 21837 ts_tablet_manager.cc:1905] T ca8de8e89fbf418bbc2ab79a53b448fe P 92eb5ec5ac944cfca7c5420a72b4652c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:44:46.520090 21836 ts_tablet_manager.cc:1905] T ca8de8e89fbf418bbc2ab79a53b448fe P b711413b9ffb4c8b9072671dd5d5f199: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:44:46.522087 21837 ts_tablet_manager.cc:1918] T ca8de8e89fbf418bbc2ab79a53b448fe P 92eb5ec5ac944cfca7c5420a72b4652c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:44:46.522099 21836 ts_tablet_manager.cc:1918] T ca8de8e89fbf418bbc2ab79a53b448fe P b711413b9ffb4c8b9072671dd5d5f199: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:44:46.522179 21837 log.cc:1196] T ca8de8e89fbf418bbc2ab79a53b448fe P 92eb5ec5ac944cfca7c5420a72b4652c: Deleting WAL directory at /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/ts-0/wal/wals/ca8de8e89fbf418bbc2ab79a53b448fe
I20260806 22:44:46.522228 21836 log.cc:1196] T ca8de8e89fbf418bbc2ab79a53b448fe P b711413b9ffb4c8b9072671dd5d5f199: Deleting WAL directory at /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/ts-2/wal/wals/ca8de8e89fbf418bbc2ab79a53b448fe
I20260806 22:44:46.522456 21837 ts_tablet_manager.cc:1939] T ca8de8e89fbf418bbc2ab79a53b448fe P 92eb5ec5ac944cfca7c5420a72b4652c: Deleting consensus metadata
I20260806 22:44:46.522552 21836 ts_tablet_manager.cc:1939] T ca8de8e89fbf418bbc2ab79a53b448fe P b711413b9ffb4c8b9072671dd5d5f199: Deleting consensus metadata
I20260806 22:44:46.523051 21049 catalog_manager.cc:4841] TS 92eb5ec5ac944cfca7c5420a72b4652c (127.18.45.193:33257): tablet ca8de8e89fbf418bbc2ab79a53b448fe (table test-table [id=a60c867d5f064f9493512807af072e15]) successfully deleted
I20260806 22:44:46.523259 21052 catalog_manager.cc:4841] TS b711413b9ffb4c8b9072671dd5d5f199 (127.18.45.195:44247): tablet ca8de8e89fbf418bbc2ab79a53b448fe (table test-table [id=a60c867d5f064f9493512807af072e15]) successfully deleted
I20260806 22:44:46.651257 21830 raft_consensus.cc:482] T 8522531482fc4eba8ad03aad8ba6c201 P af47fa363a39460f9128a46c2a601ec4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:44:46.651430 21830 raft_consensus.cc:504] T 8522531482fc4eba8ad03aad8ba6c201 P af47fa363a39460f9128a46c2a601ec4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33257 } } peers { permanent_uuid: "af47fa363a39460f9128a46c2a601ec4" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45227 } } peers { permanent_uuid: "b711413b9ffb4c8b9072671dd5d5f199" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44247 } }
I20260806 22:44:46.651727 21830 leader_election.cc:288] T 8522531482fc4eba8ad03aad8ba6c201 P af47fa363a39460f9128a46c2a601ec4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 92eb5ec5ac944cfca7c5420a72b4652c (127.18.45.193:33257), b711413b9ffb4c8b9072671dd5d5f199 (127.18.45.195:44247)
I20260806 22:44:46.652064 21288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8522531482fc4eba8ad03aad8ba6c201" candidate_uuid: "af47fa363a39460f9128a46c2a601ec4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" is_pre_election: true
I20260806 22:44:46.652072 21682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8522531482fc4eba8ad03aad8ba6c201" candidate_uuid: "af47fa363a39460f9128a46c2a601ec4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b711413b9ffb4c8b9072671dd5d5f199" is_pre_election: true
I20260806 22:44:46.652217 21682 raft_consensus.cc:2454] T 8522531482fc4eba8ad03aad8ba6c201 P b711413b9ffb4c8b9072671dd5d5f199 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af47fa363a39460f9128a46c2a601ec4 in term 0.
I20260806 22:44:46.652216 21288 raft_consensus.cc:2454] T 8522531482fc4eba8ad03aad8ba6c201 P 92eb5ec5ac944cfca7c5420a72b4652c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af47fa363a39460f9128a46c2a601ec4 in term 0.
I20260806 22:44:46.652438 21390 leader_election.cc:302] T 8522531482fc4eba8ad03aad8ba6c201 P af47fa363a39460f9128a46c2a601ec4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92eb5ec5ac944cfca7c5420a72b4652c, af47fa363a39460f9128a46c2a601ec4; no voters:
I20260806 22:44:46.652587 21830 raft_consensus.cc:2781] T 8522531482fc4eba8ad03aad8ba6c201 P af47fa363a39460f9128a46c2a601ec4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:44:46.652669 21830 raft_consensus.cc:482] T 8522531482fc4eba8ad03aad8ba6c201 P af47fa363a39460f9128a46c2a601ec4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:44:46.652699 21830 raft_consensus.cc:3037] T 8522531482fc4eba8ad03aad8ba6c201 P af47fa363a39460f9128a46c2a601ec4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:46.653616 21830 raft_consensus.cc:504] T 8522531482fc4eba8ad03aad8ba6c201 P af47fa363a39460f9128a46c2a601ec4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33257 } } peers { permanent_uuid: "af47fa363a39460f9128a46c2a601ec4" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45227 } } peers { permanent_uuid: "b711413b9ffb4c8b9072671dd5d5f199" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44247 } }
I20260806 22:44:46.653793 21830 leader_election.cc:288] T 8522531482fc4eba8ad03aad8ba6c201 P af47fa363a39460f9128a46c2a601ec4 [CANDIDATE]: Term 1 election: Requested vote from peers 92eb5ec5ac944cfca7c5420a72b4652c (127.18.45.193:33257), b711413b9ffb4c8b9072671dd5d5f199 (127.18.45.195:44247)
I20260806 22:44:46.654023 21288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8522531482fc4eba8ad03aad8ba6c201" candidate_uuid: "af47fa363a39460f9128a46c2a601ec4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92eb5ec5ac944cfca7c5420a72b4652c"
I20260806 22:44:46.654124 21288 raft_consensus.cc:3037] T 8522531482fc4eba8ad03aad8ba6c201 P 92eb5ec5ac944cfca7c5420a72b4652c [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:46.654063 21682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8522531482fc4eba8ad03aad8ba6c201" candidate_uuid: "af47fa363a39460f9128a46c2a601ec4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b711413b9ffb4c8b9072671dd5d5f199"
I20260806 22:44:46.654188 21682 raft_consensus.cc:3037] T 8522531482fc4eba8ad03aad8ba6c201 P b711413b9ffb4c8b9072671dd5d5f199 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:46.654932 21682 raft_consensus.cc:2454] T 8522531482fc4eba8ad03aad8ba6c201 P b711413b9ffb4c8b9072671dd5d5f199 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af47fa363a39460f9128a46c2a601ec4 in term 1.
I20260806 22:44:46.655194 21395 leader_election.cc:302] T 8522531482fc4eba8ad03aad8ba6c201 P af47fa363a39460f9128a46c2a601ec4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: af47fa363a39460f9128a46c2a601ec4, b711413b9ffb4c8b9072671dd5d5f199; no voters:
I20260806 22:44:46.655212 21288 raft_consensus.cc:2454] T 8522531482fc4eba8ad03aad8ba6c201 P 92eb5ec5ac944cfca7c5420a72b4652c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af47fa363a39460f9128a46c2a601ec4 in term 1.
I20260806 22:44:46.655305 21830 raft_consensus.cc:2781] T 8522531482fc4eba8ad03aad8ba6c201 P af47fa363a39460f9128a46c2a601ec4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:44:46.655366 21830 raft_consensus.cc:686] T 8522531482fc4eba8ad03aad8ba6c201 P af47fa363a39460f9128a46c2a601ec4 [term 1 LEADER]: Becoming Leader. State: Replica: af47fa363a39460f9128a46c2a601ec4, State: Running, Role: LEADER
I20260806 22:44:46.655427 21830 consensus_queue.cc:237] T 8522531482fc4eba8ad03aad8ba6c201 P af47fa363a39460f9128a46c2a601ec4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33257 } } peers { permanent_uuid: "af47fa363a39460f9128a46c2a601ec4" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45227 } } peers { permanent_uuid: "b711413b9ffb4c8b9072671dd5d5f199" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44247 } }
I20260806 22:44:46.656304 21066 catalog_manager.cc:5495] T 8522531482fc4eba8ad03aad8ba6c201 P af47fa363a39460f9128a46c2a601ec4 reported cstate change: term changed from 0 to 1, leader changed from <none> to af47fa363a39460f9128a46c2a601ec4 (127.18.45.194). New cstate: current_term: 1 leader_uuid: "af47fa363a39460f9128a46c2a601ec4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af47fa363a39460f9128a46c2a601ec4" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45227 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b711413b9ffb4c8b9072671dd5d5f199" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44247 } health_report { overall_health: UNKNOWN } } }
I20260806 22:44:46.668375 21682 raft_consensus.cc:1261] T 8522531482fc4eba8ad03aad8ba6c201 P b711413b9ffb4c8b9072671dd5d5f199 [term 1 FOLLOWER]: Refusing update from remote peer af47fa363a39460f9128a46c2a601ec4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:44:46.668591 21288 raft_consensus.cc:1261] T 8522531482fc4eba8ad03aad8ba6c201 P 92eb5ec5ac944cfca7c5420a72b4652c [term 1 FOLLOWER]: Refusing update from remote peer af47fa363a39460f9128a46c2a601ec4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:44:46.668704 21830 consensus_queue.cc:1035] T 8522531482fc4eba8ad03aad8ba6c201 P af47fa363a39460f9128a46c2a601ec4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b711413b9ffb4c8b9072671dd5d5f199" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:44:46.668833 21796 consensus_queue.cc:1035] T 8522531482fc4eba8ad03aad8ba6c201 P af47fa363a39460f9128a46c2a601ec4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 22:44:46.674500 21817 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 22:44:48.197052 21068 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47968:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:44:48.197238 21068 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47968:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:44:48.198549 21068 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e54ba47edd124325be88cc0cc4df70b5: Sending DeleteTablet for 3 replicas of tablet 8522531482fc4eba8ad03aad8ba6c201
I20260806 22:44:48.198954 21464 tablet_service.cc:1514] Processing DeleteTablet for tablet 8522531482fc4eba8ad03aad8ba6c201 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:44:48 UTC) from {username='kudu', principal='kudu/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:40484
I20260806 22:44:48.199084 21640 tablet_service.cc:1514] Processing DeleteTablet for tablet 8522531482fc4eba8ad03aad8ba6c201 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:44:48 UTC) from {username='kudu', principal='kudu/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:46594
I20260806 22:44:48.199110 21260 tablet_service.cc:1514] Processing DeleteTablet for tablet 8522531482fc4eba8ad03aad8ba6c201 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:44:48 UTC) from {username='kudu', principal='kudu/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:47084
I20260806 22:44:48.199527 21878 tablet_replica.cc:330] T 8522531482fc4eba8ad03aad8ba6c201 P b711413b9ffb4c8b9072671dd5d5f199: stopping tablet replica
I20260806 22:44:48.203599 21878 raft_consensus.cc:2229] T 8522531482fc4eba8ad03aad8ba6c201 P b711413b9ffb4c8b9072671dd5d5f199 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:44:48.203717 21878 raft_consensus.cc:2258] T 8522531482fc4eba8ad03aad8ba6c201 P b711413b9ffb4c8b9072671dd5d5f199 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:44:48.204187 21878 ts_tablet_manager.cc:1905] T 8522531482fc4eba8ad03aad8ba6c201 P b711413b9ffb4c8b9072671dd5d5f199: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:44:48.206048 21880 tablet_replica.cc:330] T 8522531482fc4eba8ad03aad8ba6c201 P 92eb5ec5ac944cfca7c5420a72b4652c: stopping tablet replica
I20260806 22:44:48.206094 21878 ts_tablet_manager.cc:1918] T 8522531482fc4eba8ad03aad8ba6c201 P b711413b9ffb4c8b9072671dd5d5f199: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 22:44:48.206156 21878 log.cc:1196] T 8522531482fc4eba8ad03aad8ba6c201 P b711413b9ffb4c8b9072671dd5d5f199: Deleting WAL directory at /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/ts-2/wal/wals/8522531482fc4eba8ad03aad8ba6c201
I20260806 22:44:48.206156 21880 raft_consensus.cc:2229] T 8522531482fc4eba8ad03aad8ba6c201 P 92eb5ec5ac944cfca7c5420a72b4652c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:44:48.206218 21880 raft_consensus.cc:2258] T 8522531482fc4eba8ad03aad8ba6c201 P 92eb5ec5ac944cfca7c5420a72b4652c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:44:48.206485 21878 ts_tablet_manager.cc:1939] T 8522531482fc4eba8ad03aad8ba6c201 P b711413b9ffb4c8b9072671dd5d5f199: Deleting consensus metadata
I20260806 22:44:48.206650 21880 ts_tablet_manager.cc:1905] T 8522531482fc4eba8ad03aad8ba6c201 P 92eb5ec5ac944cfca7c5420a72b4652c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:44:48.206986 21052 catalog_manager.cc:4841] TS b711413b9ffb4c8b9072671dd5d5f199 (127.18.45.195:44247): tablet 8522531482fc4eba8ad03aad8ba6c201 (table test-table [id=54b3a18a7f0340ab899c669a049514f1]) successfully deleted
I20260806 22:44:48.207186 21877 tablet_replica.cc:330] T 8522531482fc4eba8ad03aad8ba6c201 P af47fa363a39460f9128a46c2a601ec4: stopping tablet replica
I20260806 22:44:48.207283 21877 raft_consensus.cc:2229] T 8522531482fc4eba8ad03aad8ba6c201 P af47fa363a39460f9128a46c2a601ec4 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:44:48.207391 21877 raft_consensus.cc:2258] T 8522531482fc4eba8ad03aad8ba6c201 P af47fa363a39460f9128a46c2a601ec4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:44:48.207697 21877 ts_tablet_manager.cc:1905] T 8522531482fc4eba8ad03aad8ba6c201 P af47fa363a39460f9128a46c2a601ec4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:44:48.208827 21880 ts_tablet_manager.cc:1918] T 8522531482fc4eba8ad03aad8ba6c201 P 92eb5ec5ac944cfca7c5420a72b4652c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 22:44:48.208897 21880 log.cc:1196] T 8522531482fc4eba8ad03aad8ba6c201 P 92eb5ec5ac944cfca7c5420a72b4652c: Deleting WAL directory at /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/ts-0/wal/wals/8522531482fc4eba8ad03aad8ba6c201
I20260806 22:44:48.209187 21880 ts_tablet_manager.cc:1939] T 8522531482fc4eba8ad03aad8ba6c201 P 92eb5ec5ac944cfca7c5420a72b4652c: Deleting consensus metadata
I20260806 22:44:48.209987 21877 ts_tablet_manager.cc:1918] T 8522531482fc4eba8ad03aad8ba6c201 P af47fa363a39460f9128a46c2a601ec4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 22:44:48.210050 21877 log.cc:1196] T 8522531482fc4eba8ad03aad8ba6c201 P af47fa363a39460f9128a46c2a601ec4: Deleting WAL directory at /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/ts-1/wal/wals/8522531482fc4eba8ad03aad8ba6c201
I20260806 22:44:48.210281 21877 ts_tablet_manager.cc:1939] T 8522531482fc4eba8ad03aad8ba6c201 P af47fa363a39460f9128a46c2a601ec4: Deleting consensus metadata
I20260806 22:44:48.212203 21050 catalog_manager.cc:4841] TS af47fa363a39460f9128a46c2a601ec4 (127.18.45.194:45227): tablet 8522531482fc4eba8ad03aad8ba6c201 (table test-table [id=54b3a18a7f0340ab899c669a049514f1]) successfully deleted
I20260806 22:44:48.212365 21049 catalog_manager.cc:4841] TS 92eb5ec5ac944cfca7c5420a72b4652c (127.18.45.193:33257): tablet 8522531482fc4eba8ad03aad8ba6c201 (table test-table [id=54b3a18a7f0340ab899c669a049514f1]) successfully deleted
Aug 06 22:44:48 dist-test-slave-h3ft krb5kdc[21009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056288, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 22:44:48 dist-test-slave-h3ft krb5kdc[21009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056288, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:44:48.239735 21889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:48.226101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47996 (local address 127.18.45.254:37773)
0806 22:44:48.226308 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:48.226311 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:48.226337 (+ 26us) server_negotiation.cc:407] Connection header received
0806 22:44:48.226429 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:48.226430 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:48.226446 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:48.226533 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:48.227417 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:48.228006 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:48.228892 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:48.229127 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:48.229184 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:48.236504 (+ 7320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:48.236507 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:48.236507 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:48.236520 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:48.238289 (+ 1769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:48.238679 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:48.238680 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:48.238681 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:48.238758 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:48.239093 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:48.239095 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:48.239096 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:48.239284 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:48.239394 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:48.239533 (+ 139us) server_negotiation.cc:299] Negotiation successful
0806 22:44:48.239633 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":137,"threads_started":1}
I20260806 22:44:48.241888 21068 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47996:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:44:48.242141 21068 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:44:48.244107 21260 tablet_service.cc:1467] Processing CreateTablet for tablet 3b571369ec3042b8ac88e612f82b5e6d (DEFAULT_TABLE table=test-table [id=8839348cf903472e87ca7f6e47d442f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:44:48.244139 21464 tablet_service.cc:1467] Processing CreateTablet for tablet 3b571369ec3042b8ac88e612f82b5e6d (DEFAULT_TABLE table=test-table [id=8839348cf903472e87ca7f6e47d442f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:44:48.244145 21640 tablet_service.cc:1467] Processing CreateTablet for tablet 3b571369ec3042b8ac88e612f82b5e6d (DEFAULT_TABLE table=test-table [id=8839348cf903472e87ca7f6e47d442f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:44:48.244329 21464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b571369ec3042b8ac88e612f82b5e6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:48.244351 21260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b571369ec3042b8ac88e612f82b5e6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:48.244369 21640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b571369ec3042b8ac88e612f82b5e6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:48.246361 21893 tablet_bootstrap.cc:492] T 3b571369ec3042b8ac88e612f82b5e6d P 92eb5ec5ac944cfca7c5420a72b4652c: Bootstrap starting.
I20260806 22:44:48.246368 21890 tablet_bootstrap.cc:492] T 3b571369ec3042b8ac88e612f82b5e6d P af47fa363a39460f9128a46c2a601ec4: Bootstrap starting.
I20260806 22:44:48.246865 21893 tablet_bootstrap.cc:654] T 3b571369ec3042b8ac88e612f82b5e6d P 92eb5ec5ac944cfca7c5420a72b4652c: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:48.246894 21890 tablet_bootstrap.cc:654] T 3b571369ec3042b8ac88e612f82b5e6d P af47fa363a39460f9128a46c2a601ec4: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:48.247323 21891 tablet_bootstrap.cc:492] T 3b571369ec3042b8ac88e612f82b5e6d P b711413b9ffb4c8b9072671dd5d5f199: Bootstrap starting.
I20260806 22:44:48.247738 21893 tablet_bootstrap.cc:492] T 3b571369ec3042b8ac88e612f82b5e6d P 92eb5ec5ac944cfca7c5420a72b4652c: No bootstrap required, opened a new log
I20260806 22:44:48.247786 21891 tablet_bootstrap.cc:654] T 3b571369ec3042b8ac88e612f82b5e6d P b711413b9ffb4c8b9072671dd5d5f199: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:48.247805 21893 ts_tablet_manager.cc:1397] T 3b571369ec3042b8ac88e612f82b5e6d P 92eb5ec5ac944cfca7c5420a72b4652c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 22:44:48.247931 21890 tablet_bootstrap.cc:492] T 3b571369ec3042b8ac88e612f82b5e6d P af47fa363a39460f9128a46c2a601ec4: No bootstrap required, opened a new log
I20260806 22:44:48.248024 21890 ts_tablet_manager.cc:1397] T 3b571369ec3042b8ac88e612f82b5e6d P af47fa363a39460f9128a46c2a601ec4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:44:48.248030 21893 raft_consensus.cc:348] T 3b571369ec3042b8ac88e612f82b5e6d P 92eb5ec5ac944cfca7c5420a72b4652c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33257 } } peers { permanent_uuid: "af47fa363a39460f9128a46c2a601ec4" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45227 } } peers { permanent_uuid: "b711413b9ffb4c8b9072671dd5d5f199" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44247 } }
I20260806 22:44:48.248129 21893 raft_consensus.cc:374] T 3b571369ec3042b8ac88e612f82b5e6d P 92eb5ec5ac944cfca7c5420a72b4652c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:48.248159 21893 raft_consensus.cc:729] T 3b571369ec3042b8ac88e612f82b5e6d P 92eb5ec5ac944cfca7c5420a72b4652c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92eb5ec5ac944cfca7c5420a72b4652c, State: Initialized, Role: FOLLOWER
I20260806 22:44:48.248215 21890 raft_consensus.cc:348] T 3b571369ec3042b8ac88e612f82b5e6d P af47fa363a39460f9128a46c2a601ec4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33257 } } peers { permanent_uuid: "af47fa363a39460f9128a46c2a601ec4" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45227 } } peers { permanent_uuid: "b711413b9ffb4c8b9072671dd5d5f199" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44247 } }
I20260806 22:44:48.248291 21893 consensus_queue.cc:260] T 3b571369ec3042b8ac88e612f82b5e6d P 92eb5ec5ac944cfca7c5420a72b4652c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33257 } } peers { permanent_uuid: "af47fa363a39460f9128a46c2a601ec4" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45227 } } peers { permanent_uuid: "b711413b9ffb4c8b9072671dd5d5f199" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44247 } }
I20260806 22:44:48.248353 21890 raft_consensus.cc:374] T 3b571369ec3042b8ac88e612f82b5e6d P af47fa363a39460f9128a46c2a601ec4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:48.248392 21890 raft_consensus.cc:729] T 3b571369ec3042b8ac88e612f82b5e6d P af47fa363a39460f9128a46c2a601ec4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af47fa363a39460f9128a46c2a601ec4, State: Initialized, Role: FOLLOWER
I20260806 22:44:48.248467 21890 consensus_queue.cc:260] T 3b571369ec3042b8ac88e612f82b5e6d P af47fa363a39460f9128a46c2a601ec4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33257 } } peers { permanent_uuid: "af47fa363a39460f9128a46c2a601ec4" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45227 } } peers { permanent_uuid: "b711413b9ffb4c8b9072671dd5d5f199" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44247 } }
I20260806 22:44:48.248515 21893 ts_tablet_manager.cc:1428] T 3b571369ec3042b8ac88e612f82b5e6d P 92eb5ec5ac944cfca7c5420a72b4652c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 22:44:48.248795 21890 ts_tablet_manager.cc:1428] T 3b571369ec3042b8ac88e612f82b5e6d P af47fa363a39460f9128a46c2a601ec4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:44:48.248906 21891 tablet_bootstrap.cc:492] T 3b571369ec3042b8ac88e612f82b5e6d P b711413b9ffb4c8b9072671dd5d5f199: No bootstrap required, opened a new log
I20260806 22:44:48.248955 21891 ts_tablet_manager.cc:1397] T 3b571369ec3042b8ac88e612f82b5e6d P b711413b9ffb4c8b9072671dd5d5f199: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:44:48.249130 21891 raft_consensus.cc:348] T 3b571369ec3042b8ac88e612f82b5e6d P b711413b9ffb4c8b9072671dd5d5f199 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33257 } } peers { permanent_uuid: "af47fa363a39460f9128a46c2a601ec4" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45227 } } peers { permanent_uuid: "b711413b9ffb4c8b9072671dd5d5f199" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44247 } }
I20260806 22:44:48.249212 21891 raft_consensus.cc:374] T 3b571369ec3042b8ac88e612f82b5e6d P b711413b9ffb4c8b9072671dd5d5f199 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:48.249238 21891 raft_consensus.cc:729] T 3b571369ec3042b8ac88e612f82b5e6d P b711413b9ffb4c8b9072671dd5d5f199 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b711413b9ffb4c8b9072671dd5d5f199, State: Initialized, Role: FOLLOWER
I20260806 22:44:48.249315 21891 consensus_queue.cc:260] T 3b571369ec3042b8ac88e612f82b5e6d P b711413b9ffb4c8b9072671dd5d5f199 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33257 } } peers { permanent_uuid: "af47fa363a39460f9128a46c2a601ec4" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45227 } } peers { permanent_uuid: "b711413b9ffb4c8b9072671dd5d5f199" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44247 } }
I20260806 22:44:48.249524 21891 ts_tablet_manager.cc:1428] T 3b571369ec3042b8ac88e612f82b5e6d P b711413b9ffb4c8b9072671dd5d5f199: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:44:48.276259 21901 raft_consensus.cc:482] T 3b571369ec3042b8ac88e612f82b5e6d P 92eb5ec5ac944cfca7c5420a72b4652c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:44:48.276397 21901 raft_consensus.cc:504] T 3b571369ec3042b8ac88e612f82b5e6d P 92eb5ec5ac944cfca7c5420a72b4652c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33257 } } peers { permanent_uuid: "af47fa363a39460f9128a46c2a601ec4" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45227 } } peers { permanent_uuid: "b711413b9ffb4c8b9072671dd5d5f199" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44247 } }
I20260806 22:44:48.276916 21901 leader_election.cc:288] T 3b571369ec3042b8ac88e612f82b5e6d P 92eb5ec5ac944cfca7c5420a72b4652c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers af47fa363a39460f9128a46c2a601ec4 (127.18.45.194:45227), b711413b9ffb4c8b9072671dd5d5f199 (127.18.45.195:44247)
I20260806 22:44:48.281710 21907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:48.277282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:34239 (local address 127.18.45.195:44247)
0806 22:44:48.277501 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:48.277503 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:48.277647 (+ 144us) server_negotiation.cc:407] Connection header received
0806 22:44:48.277684 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:48.277685 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:48.277701 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:48.277798 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:44:48.278049 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:48.278918 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:48.280636 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:48.281256 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:48.281343 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:48.281356 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:48.281527 (+ 171us) server_negotiation.cc:299] Negotiation successful
0806 22:44:48.281580 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":113,"threads_started":1}
I20260806 22:44:48.281760 21905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:48.277139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.194:45227 (local address 127.18.45.193:33723)
0806 22:44:48.277479 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:48.277498 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:48.277610 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:48.277817 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:48.277818 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:48.277822 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:44:48.277954 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:48.277958 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:48.279044 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:48.279044 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:48.280648 (+ 1604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:48.280651 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:48.281474 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:48.281475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:48.281566 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:48.281568 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:44:48.281622 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 22:44:48.281655 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"mutex_wait_us":28,"thread_start_us":246,"threads_started":1}
I20260806 22:44:48.281797 21904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:48.277249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:33723 (local address 127.18.45.194:45227)
0806 22:44:48.277488 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:48.277491 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:48.277602 (+ 111us) server_negotiation.cc:407] Connection header received
0806 22:44:48.277684 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:48.277685 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:48.277700 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:48.277798 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:44:48.278058 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:48.278911 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:48.280752 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:48.281381 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:48.281450 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:48.281459 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:48.281646 (+ 187us) server_negotiation.cc:299] Negotiation successful
0806 22:44:48.281697 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":114,"threads_started":1}
I20260806 22:44:48.282085 21906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:48.277190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.195:44247 (local address 127.18.45.193:34239)
0806 22:44:48.277563 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:48.277574 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:48.277632 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:48.277816 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:48.277816 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:48.277820 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:44:48.277955 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:48.277957 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:48.279042 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:48.279044 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:48.280508 (+ 1464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:48.280529 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:48.281345 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:48.281346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:48.281407 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:48.281411 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:44:48.281489 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 22:44:48.281560 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"mutex_wait_us":6,"thread_start_us":189,"threads_started":1}
I20260806 22:44:48.282363 21682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b571369ec3042b8ac88e612f82b5e6d" candidate_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b711413b9ffb4c8b9072671dd5d5f199" is_pre_election: true
I20260806 22:44:48.282490 21682 raft_consensus.cc:2454] T 3b571369ec3042b8ac88e612f82b5e6d P b711413b9ffb4c8b9072671dd5d5f199 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92eb5ec5ac944cfca7c5420a72b4652c in term 0.
I20260806 22:44:48.282406 21489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b571369ec3042b8ac88e612f82b5e6d" candidate_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af47fa363a39460f9128a46c2a601ec4" is_pre_election: true
I20260806 22:44:48.282613 21489 raft_consensus.cc:2454] T 3b571369ec3042b8ac88e612f82b5e6d P af47fa363a39460f9128a46c2a601ec4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92eb5ec5ac944cfca7c5420a72b4652c in term 0.
I20260806 22:44:48.282739 21191 leader_election.cc:302] T 3b571369ec3042b8ac88e612f82b5e6d P 92eb5ec5ac944cfca7c5420a72b4652c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92eb5ec5ac944cfca7c5420a72b4652c, b711413b9ffb4c8b9072671dd5d5f199; no voters:
I20260806 22:44:48.282914 21901 raft_consensus.cc:2781] T 3b571369ec3042b8ac88e612f82b5e6d P 92eb5ec5ac944cfca7c5420a72b4652c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:44:48.282958 21901 raft_consensus.cc:482] T 3b571369ec3042b8ac88e612f82b5e6d P 92eb5ec5ac944cfca7c5420a72b4652c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:44:48.282979 21901 raft_consensus.cc:3037] T 3b571369ec3042b8ac88e612f82b5e6d P 92eb5ec5ac944cfca7c5420a72b4652c [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:48.283927 21901 raft_consensus.cc:504] T 3b571369ec3042b8ac88e612f82b5e6d P 92eb5ec5ac944cfca7c5420a72b4652c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33257 } } peers { permanent_uuid: "af47fa363a39460f9128a46c2a601ec4" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45227 } } peers { permanent_uuid: "b711413b9ffb4c8b9072671dd5d5f199" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44247 } }
I20260806 22:44:48.284107 21901 leader_election.cc:288] T 3b571369ec3042b8ac88e612f82b5e6d P 92eb5ec5ac944cfca7c5420a72b4652c [CANDIDATE]: Term 1 election: Requested vote from peers af47fa363a39460f9128a46c2a601ec4 (127.18.45.194:45227), b711413b9ffb4c8b9072671dd5d5f199 (127.18.45.195:44247)
I20260806 22:44:48.284286 21489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b571369ec3042b8ac88e612f82b5e6d" candidate_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af47fa363a39460f9128a46c2a601ec4"
I20260806 22:44:48.284296 21682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b571369ec3042b8ac88e612f82b5e6d" candidate_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b711413b9ffb4c8b9072671dd5d5f199"
I20260806 22:44:48.284375 21489 raft_consensus.cc:3037] T 3b571369ec3042b8ac88e612f82b5e6d P af47fa363a39460f9128a46c2a601ec4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:48.284377 21682 raft_consensus.cc:3037] T 3b571369ec3042b8ac88e612f82b5e6d P b711413b9ffb4c8b9072671dd5d5f199 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:48.285319 21682 raft_consensus.cc:2454] T 3b571369ec3042b8ac88e612f82b5e6d P b711413b9ffb4c8b9072671dd5d5f199 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92eb5ec5ac944cfca7c5420a72b4652c in term 1.
I20260806 22:44:48.285422 21489 raft_consensus.cc:2454] T 3b571369ec3042b8ac88e612f82b5e6d P af47fa363a39460f9128a46c2a601ec4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92eb5ec5ac944cfca7c5420a72b4652c in term 1.
I20260806 22:44:48.285519 21191 leader_election.cc:302] T 3b571369ec3042b8ac88e612f82b5e6d P 92eb5ec5ac944cfca7c5420a72b4652c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92eb5ec5ac944cfca7c5420a72b4652c, b711413b9ffb4c8b9072671dd5d5f199; no voters:
I20260806 22:44:48.285612 21901 raft_consensus.cc:2781] T 3b571369ec3042b8ac88e612f82b5e6d P 92eb5ec5ac944cfca7c5420a72b4652c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:44:48.285825 21901 raft_consensus.cc:686] T 3b571369ec3042b8ac88e612f82b5e6d P 92eb5ec5ac944cfca7c5420a72b4652c [term 1 LEADER]: Becoming Leader. State: Replica: 92eb5ec5ac944cfca7c5420a72b4652c, State: Running, Role: LEADER
I20260806 22:44:48.285913 21901 consensus_queue.cc:237] T 3b571369ec3042b8ac88e612f82b5e6d P 92eb5ec5ac944cfca7c5420a72b4652c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33257 } } peers { permanent_uuid: "af47fa363a39460f9128a46c2a601ec4" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45227 } } peers { permanent_uuid: "b711413b9ffb4c8b9072671dd5d5f199" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44247 } }
I20260806 22:44:48.286752 21068 catalog_manager.cc:5495] T 3b571369ec3042b8ac88e612f82b5e6d P 92eb5ec5ac944cfca7c5420a72b4652c reported cstate change: term changed from 0 to 1, leader changed from <none> to 92eb5ec5ac944cfca7c5420a72b4652c (127.18.45.193). New cstate: current_term: 1 leader_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af47fa363a39460f9128a46c2a601ec4" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b711413b9ffb4c8b9072671dd5d5f199" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44247 } health_report { overall_health: UNKNOWN } } }
I20260806 22:44:48.300284 21911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:48.297265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47098 (local address 127.18.45.193:33257)
0806 22:44:48.297485 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:48.297488 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:48.297496 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:44:48.297548 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:48.297548 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:48.297578 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:48.297671 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:44:48.297886 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:48.298471 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:48.299368 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:48.299620 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:48.299691 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:48.299818 (+ 127us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:44:48.299976 (+ 158us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:44:48.300035 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:48.300091 (+ 56us) server_negotiation.cc:299] Negotiation successful
0806 22:44:48.300117 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":122,"threads_started":1}
I20260806 22:44:48.306515 21904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:48.303696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:60711 (local address 127.18.45.194:45227)
0806 22:44:48.303781 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:48.303783 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:48.303958 (+ 175us) server_negotiation.cc:407] Connection header received
0806 22:44:48.304024 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:48.304025 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:48.304037 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:48.304097 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:44:48.304277 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:48.304791 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:48.305688 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:48.305891 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:48.305956 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:48.306194 (+ 238us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:44:48.306235 (+ 41us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:44:48.306286 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:48.306400 (+ 114us) server_negotiation.cc:299] Negotiation successful
0806 22:44:48.306431 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 22:44:48.306531 21914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:48.303522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.194:45227 (local address 127.18.45.193:60711)
0806 22:44:48.303878 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:48.303891 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:48.303961 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:48.304114 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:48.304114 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:48.304117 (+ 3us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:44:48.304214 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:48.304216 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:48.304917 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:48.304918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:48.305605 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:48.305607 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:48.305992 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:48.305993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:48.306061 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:48.306080 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:44:48.306320 (+ 240us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:44:48.306323 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:44:48.306386 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 22:44:48.306441 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":211,"threads_started":1}
I20260806 22:44:48.309851 21682 raft_consensus.cc:1261] T 3b571369ec3042b8ac88e612f82b5e6d P b711413b9ffb4c8b9072671dd5d5f199 [term 1 FOLLOWER]: Refusing update from remote peer 92eb5ec5ac944cfca7c5420a72b4652c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:44:48.309962 21489 raft_consensus.cc:1261] T 3b571369ec3042b8ac88e612f82b5e6d P af47fa363a39460f9128a46c2a601ec4 [term 1 FOLLOWER]: Refusing update from remote peer 92eb5ec5ac944cfca7c5420a72b4652c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:44:48.310196 21908 consensus_queue.cc:1035] T 3b571369ec3042b8ac88e612f82b5e6d P 92eb5ec5ac944cfca7c5420a72b4652c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b711413b9ffb4c8b9072671dd5d5f199" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:44:48.310429 21901 consensus_queue.cc:1035] T 3b571369ec3042b8ac88e612f82b5e6d P 92eb5ec5ac944cfca7c5420a72b4652c [LEADER]: Connected to new peer: Peer: permanent_uuid: "af47fa363a39460f9128a46c2a601ec4" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:44:48.319497 21924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:48.316677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.193:33257 (local address 127.18.45.194:58637)
0806 22:44:48.316829 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:48.316841 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:48.316912 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:48.317048 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:48.317049 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:48.317050 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:44:48.317112 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:48.317113 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:48.317737 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:48.317738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:48.318434 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:48.318437 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:48.319078 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:48.319079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:48.319155 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:48.319162 (+ 7us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:44:48.319350 (+ 188us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:44:48.319351 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 22:44:48.319378 (+ 27us) client_negotiation.cc:240] Negotiation successful
0806 22:44:48.319427 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":100,"threads_started":1}
I20260806 22:44:48.319491 21911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:48.316732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:58637 (local address 127.18.45.193:33257)
0806 22:44:48.316780 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:48.316782 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:48.316901 (+ 119us) server_negotiation.cc:407] Connection header received
0806 22:44:48.316968 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:48.316969 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:48.316983 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:48.317039 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:44:48.317162 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:48.317669 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:48.318623 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:48.318940 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:48.319052 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:48.319246 (+ 194us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:44:48.319284 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:44:48.319350 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:48.319390 (+ 40us) server_negotiation.cc:299] Negotiation successful
0806 22:44:48.319416 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:44:48.330149 21068 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47996:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:44:48.330317 21068 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47996:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:44:48.331275 21068 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e54ba47edd124325be88cc0cc4df70b5: Sending DeleteTablet for 3 replicas of tablet 3b571369ec3042b8ac88e612f82b5e6d
I20260806 22:44:48.331562 21260 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b571369ec3042b8ac88e612f82b5e6d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:44:48 UTC) from {username='kudu', principal='kudu/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:47084
I20260806 22:44:48.331624 21640 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b571369ec3042b8ac88e612f82b5e6d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:44:48 UTC) from {username='kudu', principal='kudu/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:46594
I20260806 22:44:48.331688 21880 tablet_replica.cc:330] T 3b571369ec3042b8ac88e612f82b5e6d P 92eb5ec5ac944cfca7c5420a72b4652c: stopping tablet replica
I20260806 22:44:48.331745 21880 raft_consensus.cc:2229] T 3b571369ec3042b8ac88e612f82b5e6d P 92eb5ec5ac944cfca7c5420a72b4652c [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:44:48.331743 21878 tablet_replica.cc:330] T 3b571369ec3042b8ac88e612f82b5e6d P b711413b9ffb4c8b9072671dd5d5f199: stopping tablet replica
I20260806 22:44:48.331825 21878 raft_consensus.cc:2229] T 3b571369ec3042b8ac88e612f82b5e6d P b711413b9ffb4c8b9072671dd5d5f199 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:44:48.331861 21880 raft_consensus.cc:2258] T 3b571369ec3042b8ac88e612f82b5e6d P 92eb5ec5ac944cfca7c5420a72b4652c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:44:48.331892 21878 raft_consensus.cc:2258] T 3b571369ec3042b8ac88e612f82b5e6d P b711413b9ffb4c8b9072671dd5d5f199 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:44:48.331938 21068 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47996:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:44:48.332129 21068 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:44:48.332736 21878 ts_tablet_manager.cc:1905] T 3b571369ec3042b8ac88e612f82b5e6d P b711413b9ffb4c8b9072671dd5d5f199: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:44:48.335386 21878 ts_tablet_manager.cc:1918] T 3b571369ec3042b8ac88e612f82b5e6d P b711413b9ffb4c8b9072671dd5d5f199: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:44:48.335455 21878 log.cc:1196] T 3b571369ec3042b8ac88e612f82b5e6d P b711413b9ffb4c8b9072671dd5d5f199: Deleting WAL directory at /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/ts-2/wal/wals/3b571369ec3042b8ac88e612f82b5e6d
I20260806 22:44:48.335716 21878 ts_tablet_manager.cc:1939] T 3b571369ec3042b8ac88e612f82b5e6d P b711413b9ffb4c8b9072671dd5d5f199: Deleting consensus metadata
I20260806 22:44:48.336191 21052 catalog_manager.cc:4841] TS b711413b9ffb4c8b9072671dd5d5f199 (127.18.45.195:44247): tablet 3b571369ec3042b8ac88e612f82b5e6d (table test-table [id=8839348cf903472e87ca7f6e47d442f2]) successfully deleted
I20260806 22:44:48.336288 21640 tablet_service.cc:1467] Processing CreateTablet for tablet dbf5385ee62244c496208caba93ba48d (DEFAULT_TABLE table=test-table [id=b421a5ba4dae47b7a0d1440e1bbce21b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:44:48.336408 21640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbf5385ee62244c496208caba93ba48d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:48.336917 21260 tablet_service.cc:1467] Processing CreateTablet for tablet dbf5385ee62244c496208caba93ba48d (DEFAULT_TABLE table=test-table [id=b421a5ba4dae47b7a0d1440e1bbce21b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:44:48.337098 21260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbf5385ee62244c496208caba93ba48d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:48.337812 21891 tablet_bootstrap.cc:492] T dbf5385ee62244c496208caba93ba48d P b711413b9ffb4c8b9072671dd5d5f199: Bootstrap starting.
I20260806 22:44:48.338409 21891 tablet_bootstrap.cc:654] T dbf5385ee62244c496208caba93ba48d P b711413b9ffb4c8b9072671dd5d5f199: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:48.338701 21893 tablet_bootstrap.cc:492] T dbf5385ee62244c496208caba93ba48d P 92eb5ec5ac944cfca7c5420a72b4652c: Bootstrap starting.
I20260806 22:44:48.339236 21893 tablet_bootstrap.cc:654] T dbf5385ee62244c496208caba93ba48d P 92eb5ec5ac944cfca7c5420a72b4652c: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:48.339401 21880 ts_tablet_manager.cc:1905] T 3b571369ec3042b8ac88e612f82b5e6d P 92eb5ec5ac944cfca7c5420a72b4652c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:44:48.340056 21893 tablet_bootstrap.cc:492] T dbf5385ee62244c496208caba93ba48d P 92eb5ec5ac944cfca7c5420a72b4652c: No bootstrap required, opened a new log
I20260806 22:44:48.340134 21893 ts_tablet_manager.cc:1397] T dbf5385ee62244c496208caba93ba48d P 92eb5ec5ac944cfca7c5420a72b4652c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 22:44:48.340333 21893 raft_consensus.cc:348] T dbf5385ee62244c496208caba93ba48d P 92eb5ec5ac944cfca7c5420a72b4652c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33257 } } peers { permanent_uuid: "af47fa363a39460f9128a46c2a601ec4" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45227 } } peers { permanent_uuid: "b711413b9ffb4c8b9072671dd5d5f199" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44247 } }
I20260806 22:44:48.340425 21893 raft_consensus.cc:374] T dbf5385ee62244c496208caba93ba48d P 92eb5ec5ac944cfca7c5420a72b4652c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:48.340456 21893 raft_consensus.cc:729] T dbf5385ee62244c496208caba93ba48d P 92eb5ec5ac944cfca7c5420a72b4652c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92eb5ec5ac944cfca7c5420a72b4652c, State: Initialized, Role: FOLLOWER
I20260806 22:44:48.340502 21893 consensus_queue.cc:260] T dbf5385ee62244c496208caba93ba48d P 92eb5ec5ac944cfca7c5420a72b4652c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33257 } } peers { permanent_uuid: "af47fa363a39460f9128a46c2a601ec4" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45227 } } peers { permanent_uuid: "b711413b9ffb4c8b9072671dd5d5f199" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44247 } }
I20260806 22:44:48.340659 21893 ts_tablet_manager.cc:1428] T dbf5385ee62244c496208caba93ba48d P 92eb5ec5ac944cfca7c5420a72b4652c: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260806 22:44:48.341055 21880 ts_tablet_manager.cc:1918] T 3b571369ec3042b8ac88e612f82b5e6d P 92eb5ec5ac944cfca7c5420a72b4652c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:44:48.341128 21880 log.cc:1196] T 3b571369ec3042b8ac88e612f82b5e6d P 92eb5ec5ac944cfca7c5420a72b4652c: Deleting WAL directory at /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/ts-0/wal/wals/3b571369ec3042b8ac88e612f82b5e6d
I20260806 22:44:48.341365 21880 ts_tablet_manager.cc:1939] T 3b571369ec3042b8ac88e612f82b5e6d P 92eb5ec5ac944cfca7c5420a72b4652c: Deleting consensus metadata
I20260806 22:44:48.341640 21464 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b571369ec3042b8ac88e612f82b5e6d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:44:48 UTC) from {username='kudu', principal='kudu/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:40484
I20260806 22:44:48.341740 21462 tablet_service.cc:1467] Processing CreateTablet for tablet dbf5385ee62244c496208caba93ba48d (DEFAULT_TABLE table=test-table [id=b421a5ba4dae47b7a0d1440e1bbce21b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:44:48.341790 21877 tablet_replica.cc:330] T 3b571369ec3042b8ac88e612f82b5e6d P af47fa363a39460f9128a46c2a601ec4: stopping tablet replica
I20260806 22:44:48.341853 21877 raft_consensus.cc:2229] T 3b571369ec3042b8ac88e612f82b5e6d P af47fa363a39460f9128a46c2a601ec4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:44:48.341881 21462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbf5385ee62244c496208caba93ba48d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:48.341917 21877 raft_consensus.cc:2258] T 3b571369ec3042b8ac88e612f82b5e6d P af47fa363a39460f9128a46c2a601ec4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:44:48.342267 21877 ts_tablet_manager.cc:1905] T 3b571369ec3042b8ac88e612f82b5e6d P af47fa363a39460f9128a46c2a601ec4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:44:48.343756 21890 tablet_bootstrap.cc:492] T dbf5385ee62244c496208caba93ba48d P af47fa363a39460f9128a46c2a601ec4: Bootstrap starting.
I20260806 22:44:48.343994 21877 ts_tablet_manager.cc:1918] T 3b571369ec3042b8ac88e612f82b5e6d P af47fa363a39460f9128a46c2a601ec4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:44:48.344071 21877 log.cc:1196] T 3b571369ec3042b8ac88e612f82b5e6d P af47fa363a39460f9128a46c2a601ec4: Deleting WAL directory at /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/ts-1/wal/wals/3b571369ec3042b8ac88e612f82b5e6d
I20260806 22:44:48.344144 21049 catalog_manager.cc:4841] TS 92eb5ec5ac944cfca7c5420a72b4652c (127.18.45.193:33257): tablet 3b571369ec3042b8ac88e612f82b5e6d (table test-table [id=8839348cf903472e87ca7f6e47d442f2]) successfully deleted
I20260806 22:44:48.344188 21890 tablet_bootstrap.cc:654] T dbf5385ee62244c496208caba93ba48d P af47fa363a39460f9128a46c2a601ec4: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:48.344365 21877 ts_tablet_manager.cc:1939] T 3b571369ec3042b8ac88e612f82b5e6d P af47fa363a39460f9128a46c2a601ec4: Deleting consensus metadata
I20260806 22:44:48.344766 21050 catalog_manager.cc:4841] TS af47fa363a39460f9128a46c2a601ec4 (127.18.45.194:45227): tablet 3b571369ec3042b8ac88e612f82b5e6d (table test-table [id=8839348cf903472e87ca7f6e47d442f2]) successfully deleted
I20260806 22:44:48.345017 21891 tablet_bootstrap.cc:492] T dbf5385ee62244c496208caba93ba48d P b711413b9ffb4c8b9072671dd5d5f199: No bootstrap required, opened a new log
I20260806 22:44:48.345077 21891 ts_tablet_manager.cc:1397] T dbf5385ee62244c496208caba93ba48d P b711413b9ffb4c8b9072671dd5d5f199: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260806 22:44:48.345292 21890 tablet_bootstrap.cc:492] T dbf5385ee62244c496208caba93ba48d P af47fa363a39460f9128a46c2a601ec4: No bootstrap required, opened a new log
I20260806 22:44:48.345336 21890 ts_tablet_manager.cc:1397] T dbf5385ee62244c496208caba93ba48d P af47fa363a39460f9128a46c2a601ec4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:44:48.345331 21891 raft_consensus.cc:348] T dbf5385ee62244c496208caba93ba48d P b711413b9ffb4c8b9072671dd5d5f199 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33257 } } peers { permanent_uuid: "af47fa363a39460f9128a46c2a601ec4" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45227 } } peers { permanent_uuid: "b711413b9ffb4c8b9072671dd5d5f199" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44247 } }
I20260806 22:44:48.345460 21891 raft_consensus.cc:374] T dbf5385ee62244c496208caba93ba48d P b711413b9ffb4c8b9072671dd5d5f199 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:48.345484 21891 raft_consensus.cc:729] T dbf5385ee62244c496208caba93ba48d P b711413b9ffb4c8b9072671dd5d5f199 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b711413b9ffb4c8b9072671dd5d5f199, State: Initialized, Role: FOLLOWER
I20260806 22:44:48.345490 21890 raft_consensus.cc:348] T dbf5385ee62244c496208caba93ba48d P af47fa363a39460f9128a46c2a601ec4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33257 } } peers { permanent_uuid: "af47fa363a39460f9128a46c2a601ec4" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45227 } } peers { permanent_uuid: "b711413b9ffb4c8b9072671dd5d5f199" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44247 } }
I20260806 22:44:48.345542 21891 consensus_queue.cc:260] T dbf5385ee62244c496208caba93ba48d P b711413b9ffb4c8b9072671dd5d5f199 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33257 } } peers { permanent_uuid: "af47fa363a39460f9128a46c2a601ec4" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45227 } } peers { permanent_uuid: "b711413b9ffb4c8b9072671dd5d5f199" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44247 } }
I20260806 22:44:48.345582 21890 raft_consensus.cc:374] T dbf5385ee62244c496208caba93ba48d P af47fa363a39460f9128a46c2a601ec4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:48.345608 21890 raft_consensus.cc:729] T dbf5385ee62244c496208caba93ba48d P af47fa363a39460f9128a46c2a601ec4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af47fa363a39460f9128a46c2a601ec4, State: Initialized, Role: FOLLOWER
I20260806 22:44:48.345654 21891 ts_tablet_manager.cc:1428] T dbf5385ee62244c496208caba93ba48d P b711413b9ffb4c8b9072671dd5d5f199: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:44:48.345647 21890 consensus_queue.cc:260] T dbf5385ee62244c496208caba93ba48d P af47fa363a39460f9128a46c2a601ec4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33257 } } peers { permanent_uuid: "af47fa363a39460f9128a46c2a601ec4" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45227 } } peers { permanent_uuid: "b711413b9ffb4c8b9072671dd5d5f199" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44247 } }
I20260806 22:44:48.345757 21890 ts_tablet_manager.cc:1428] T dbf5385ee62244c496208caba93ba48d P af47fa363a39460f9128a46c2a601ec4: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 22:44:48.427349 21901 raft_consensus.cc:482] T dbf5385ee62244c496208caba93ba48d P 92eb5ec5ac944cfca7c5420a72b4652c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:44:48.427500 21901 raft_consensus.cc:504] T dbf5385ee62244c496208caba93ba48d P 92eb5ec5ac944cfca7c5420a72b4652c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33257 } } peers { permanent_uuid: "af47fa363a39460f9128a46c2a601ec4" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45227 } } peers { permanent_uuid: "b711413b9ffb4c8b9072671dd5d5f199" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44247 } }
I20260806 22:44:48.427750 21901 leader_election.cc:288] T dbf5385ee62244c496208caba93ba48d P 92eb5ec5ac944cfca7c5420a72b4652c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers af47fa363a39460f9128a46c2a601ec4 (127.18.45.194:45227), b711413b9ffb4c8b9072671dd5d5f199 (127.18.45.195:44247)
I20260806 22:44:48.428081 21489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbf5385ee62244c496208caba93ba48d" candidate_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af47fa363a39460f9128a46c2a601ec4" is_pre_election: true
I20260806 22:44:48.428122 21681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbf5385ee62244c496208caba93ba48d" candidate_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b711413b9ffb4c8b9072671dd5d5f199" is_pre_election: true
I20260806 22:44:48.428228 21489 raft_consensus.cc:2454] T dbf5385ee62244c496208caba93ba48d P af47fa363a39460f9128a46c2a601ec4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92eb5ec5ac944cfca7c5420a72b4652c in term 0.
I20260806 22:44:48.428282 21681 raft_consensus.cc:2454] T dbf5385ee62244c496208caba93ba48d P b711413b9ffb4c8b9072671dd5d5f199 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92eb5ec5ac944cfca7c5420a72b4652c in term 0.
I20260806 22:44:48.428489 21188 leader_election.cc:302] T dbf5385ee62244c496208caba93ba48d P 92eb5ec5ac944cfca7c5420a72b4652c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92eb5ec5ac944cfca7c5420a72b4652c, af47fa363a39460f9128a46c2a601ec4; no voters:
I20260806 22:44:48.428622 21901 raft_consensus.cc:2781] T dbf5385ee62244c496208caba93ba48d P 92eb5ec5ac944cfca7c5420a72b4652c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:44:48.428679 21901 raft_consensus.cc:482] T dbf5385ee62244c496208caba93ba48d P 92eb5ec5ac944cfca7c5420a72b4652c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:44:48.428695 21901 raft_consensus.cc:3037] T dbf5385ee62244c496208caba93ba48d P 92eb5ec5ac944cfca7c5420a72b4652c [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:48.429811 21901 raft_consensus.cc:504] T dbf5385ee62244c496208caba93ba48d P 92eb5ec5ac944cfca7c5420a72b4652c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33257 } } peers { permanent_uuid: "af47fa363a39460f9128a46c2a601ec4" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45227 } } peers { permanent_uuid: "b711413b9ffb4c8b9072671dd5d5f199" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44247 } }
I20260806 22:44:48.430027 21901 leader_election.cc:288] T dbf5385ee62244c496208caba93ba48d P 92eb5ec5ac944cfca7c5420a72b4652c [CANDIDATE]: Term 1 election: Requested vote from peers af47fa363a39460f9128a46c2a601ec4 (127.18.45.194:45227), b711413b9ffb4c8b9072671dd5d5f199 (127.18.45.195:44247)
I20260806 22:44:48.430307 21681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbf5385ee62244c496208caba93ba48d" candidate_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b711413b9ffb4c8b9072671dd5d5f199"
I20260806 22:44:48.430330 21489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbf5385ee62244c496208caba93ba48d" candidate_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af47fa363a39460f9128a46c2a601ec4"
I20260806 22:44:48.430444 21681 raft_consensus.cc:3037] T dbf5385ee62244c496208caba93ba48d P b711413b9ffb4c8b9072671dd5d5f199 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:48.430456 21489 raft_consensus.cc:3037] T dbf5385ee62244c496208caba93ba48d P af47fa363a39460f9128a46c2a601ec4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:48.431568 21681 raft_consensus.cc:2454] T dbf5385ee62244c496208caba93ba48d P b711413b9ffb4c8b9072671dd5d5f199 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92eb5ec5ac944cfca7c5420a72b4652c in term 1.
I20260806 22:44:48.431792 21191 leader_election.cc:302] T dbf5385ee62244c496208caba93ba48d P 92eb5ec5ac944cfca7c5420a72b4652c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92eb5ec5ac944cfca7c5420a72b4652c, b711413b9ffb4c8b9072671dd5d5f199; no voters:
I20260806 22:44:48.431794 21489 raft_consensus.cc:2454] T dbf5385ee62244c496208caba93ba48d P af47fa363a39460f9128a46c2a601ec4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92eb5ec5ac944cfca7c5420a72b4652c in term 1.
I20260806 22:44:48.431918 21901 raft_consensus.cc:2781] T dbf5385ee62244c496208caba93ba48d P 92eb5ec5ac944cfca7c5420a72b4652c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:44:48.431986 21901 raft_consensus.cc:686] T dbf5385ee62244c496208caba93ba48d P 92eb5ec5ac944cfca7c5420a72b4652c [term 1 LEADER]: Becoming Leader. State: Replica: 92eb5ec5ac944cfca7c5420a72b4652c, State: Running, Role: LEADER
I20260806 22:44:48.432075 21901 consensus_queue.cc:237] T dbf5385ee62244c496208caba93ba48d P 92eb5ec5ac944cfca7c5420a72b4652c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33257 } } peers { permanent_uuid: "af47fa363a39460f9128a46c2a601ec4" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45227 } } peers { permanent_uuid: "b711413b9ffb4c8b9072671dd5d5f199" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44247 } }
I20260806 22:44:48.433045 21066 catalog_manager.cc:5495] T dbf5385ee62244c496208caba93ba48d P 92eb5ec5ac944cfca7c5420a72b4652c reported cstate change: term changed from 0 to 1, leader changed from <none> to 92eb5ec5ac944cfca7c5420a72b4652c (127.18.45.193). New cstate: current_term: 1 leader_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92eb5ec5ac944cfca7c5420a72b4652c" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 33257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af47fa363a39460f9128a46c2a601ec4" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b711413b9ffb4c8b9072671dd5d5f199" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44247 } health_report { overall_health: UNKNOWN } } }
I20260806 22:44:48.442647 21681 raft_consensus.cc:1261] T dbf5385ee62244c496208caba93ba48d P b711413b9ffb4c8b9072671dd5d5f199 [term 1 FOLLOWER]: Refusing update from remote peer 92eb5ec5ac944cfca7c5420a72b4652c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:44:48.442713 21489 raft_consensus.cc:1261] T dbf5385ee62244c496208caba93ba48d P af47fa363a39460f9128a46c2a601ec4 [term 1 FOLLOWER]: Refusing update from remote peer 92eb5ec5ac944cfca7c5420a72b4652c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:44:48.442950 21901 consensus_queue.cc:1035] T dbf5385ee62244c496208caba93ba48d P 92eb5ec5ac944cfca7c5420a72b4652c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b711413b9ffb4c8b9072671dd5d5f199" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:44:48.443107 21919 consensus_queue.cc:1035] T dbf5385ee62244c496208caba93ba48d P 92eb5ec5ac944cfca7c5420a72b4652c [LEADER]: Connected to new peer: Peer: permanent_uuid: "af47fa363a39460f9128a46c2a601ec4" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 22:44:48.448840 21855 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 22:44:49.970088 21065 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47996:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:44:49.970243 21065 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47996:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:44:49.971493 21065 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e54ba47edd124325be88cc0cc4df70b5: Sending DeleteTablet for 3 replicas of tablet dbf5385ee62244c496208caba93ba48d
I20260806 22:44:49.971910 21260 tablet_service.cc:1514] Processing DeleteTablet for tablet dbf5385ee62244c496208caba93ba48d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:44:49 UTC) from {username='kudu', principal='kudu/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:47084
I20260806 22:44:49.971987 21462 tablet_service.cc:1514] Processing DeleteTablet for tablet dbf5385ee62244c496208caba93ba48d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:44:49 UTC) from {username='kudu', principal='kudu/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:40484
I20260806 22:44:49.971962 21640 tablet_service.cc:1514] Processing DeleteTablet for tablet dbf5385ee62244c496208caba93ba48d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:44:49 UTC) from {username='kudu', principal='kudu/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:46594
I20260806 22:44:49.972209 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 21164
I20260806 22:44:49.980436 21969 tablet_replica.cc:330] T dbf5385ee62244c496208caba93ba48d P b711413b9ffb4c8b9072671dd5d5f199: stopping tablet replica
I20260806 22:44:49.980587 21969 raft_consensus.cc:2229] T dbf5385ee62244c496208caba93ba48d P b711413b9ffb4c8b9072671dd5d5f199 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:44:49.980682 21969 raft_consensus.cc:2258] T dbf5385ee62244c496208caba93ba48d P b711413b9ffb4c8b9072671dd5d5f199 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:44:49.980875 21970 tablet_replica.cc:330] T dbf5385ee62244c496208caba93ba48d P af47fa363a39460f9128a46c2a601ec4: stopping tablet replica
I20260806 22:44:49.980969 21970 raft_consensus.cc:2229] T dbf5385ee62244c496208caba93ba48d P af47fa363a39460f9128a46c2a601ec4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:44:49.981041 21970 raft_consensus.cc:2258] T dbf5385ee62244c496208caba93ba48d P af47fa363a39460f9128a46c2a601ec4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:44:49.981113 21969 ts_tablet_manager.cc:1905] T dbf5385ee62244c496208caba93ba48d P b711413b9ffb4c8b9072671dd5d5f199: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:44:49.981401 21970 ts_tablet_manager.cc:1905] T dbf5385ee62244c496208caba93ba48d P af47fa363a39460f9128a46c2a601ec4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:44:49.983147 21970 ts_tablet_manager.cc:1918] T dbf5385ee62244c496208caba93ba48d P af47fa363a39460f9128a46c2a601ec4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 22:44:49.983238 21970 log.cc:1196] T dbf5385ee62244c496208caba93ba48d P af47fa363a39460f9128a46c2a601ec4: Deleting WAL directory at /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/ts-1/wal/wals/dbf5385ee62244c496208caba93ba48d
I20260806 22:44:49.983525 21970 ts_tablet_manager.cc:1939] T dbf5385ee62244c496208caba93ba48d P af47fa363a39460f9128a46c2a601ec4: Deleting consensus metadata
I20260806 22:44:49.983578 21969 ts_tablet_manager.cc:1918] T dbf5385ee62244c496208caba93ba48d P b711413b9ffb4c8b9072671dd5d5f199: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 22:44:49.983661 21969 log.cc:1196] T dbf5385ee62244c496208caba93ba48d P b711413b9ffb4c8b9072671dd5d5f199: Deleting WAL directory at /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786056275803658-18615-0/minicluster-data/ts-2/wal/wals/dbf5385ee62244c496208caba93ba48d
I20260806 22:44:49.983908 21969 ts_tablet_manager.cc:1939] T dbf5385ee62244c496208caba93ba48d P b711413b9ffb4c8b9072671dd5d5f199: Deleting consensus metadata
I20260806 22:44:49.984117 21050 catalog_manager.cc:4841] TS af47fa363a39460f9128a46c2a601ec4 (127.18.45.194:45227): tablet dbf5385ee62244c496208caba93ba48d (table test-table [id=b421a5ba4dae47b7a0d1440e1bbce21b]) successfully deleted
I20260806 22:44:49.984566 21052 catalog_manager.cc:4841] TS b711413b9ffb4c8b9072671dd5d5f199 (127.18.45.195:44247): tablet dbf5385ee62244c496208caba93ba48d (table test-table [id=b421a5ba4dae47b7a0d1440e1bbce21b]) successfully deleted
W20260806 22:44:49.990381 21049 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.45.193:33257 (error 108)
I20260806 22:44:49.991392 21973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:49.991133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.193:33257 (local address 127.0.0.1:54246)
0806 22:44:49.991287 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:49.991339 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.193:33257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":86,"threads_started":1}
I20260806 22:44:49.991406 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 21365
W20260806 22:44:49.991533 21049 catalog_manager.cc:4584] TS 92eb5ec5ac944cfca7c5420a72b4652c (127.18.45.193:33257): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet dbf5385ee62244c496208caba93ba48d: Network error: Client connection negotiation failed: client connection to 127.18.45.193:33257: connect: Connection refused (error 111)
I20260806 22:44:50.012003 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 21565
I20260806 22:44:50.027426 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 21026
2026-08-06T22:44:50Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6253 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:44:50 dist-test-slave-h3ft krb5kdc[21976](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:44:50 dist-test-slave-h3ft krb5kdc[21976](info): set up 2 sockets
Aug 06 22:44:50 dist-test-slave-h3ft krb5kdc[21976](info): commencing operation
krb5kdc: starting...
W20260806 22:44:51.315654 18615 mini_kdc.cc:121] Time spent starting KDC: real 1.252s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Aug 06 22:44:51 dist-test-slave-h3ft krb5kdc[21976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056291, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:44:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:44:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.45.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab.
Entry for principal kudu/127.18.45.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab.
I20260806 22:44:51.461949 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:41579
--webserver_interface=127.18.45.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:45837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.45.254:41579
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:44:51.558518 21995 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:51.558836 21995 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:51.558879 21995 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:51.560218 21995 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:44:51.560308 21995 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:51.560336 21995 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:51.560359 21995 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:44:51.560388 21995 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:44:51.562151 21995 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:45837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.45.254:41579
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:41579
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.45.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21995
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:44:51.562500 21995 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:51.562800 21995 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:51.568939 22005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:51.569149 22001 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:51.569231 22000 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:51.569278 21995 server_base.cc:1034] running on GCE node
I20260806 22:44:51.569591 21995 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:51.570005 21995 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:51.571180 21995 hybrid_clock.cc:648] HybridClock initialized: now 1786056291571150 us; error 53 us; skew 500 ppm
Aug 06 22:44:51 dist-test-slave-h3ft krb5kdc[21976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056291, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:51.574363 21995 init.cc:377] Logged in from keytab as kudu/127.18.45.254@KRBTEST.COM (short username kudu)
I20260806 22:44:51.574971 21995 webserver.cc:460] Webserver started at http://127.18.45.254:35641/ using document root <none> and password file <none>
I20260806 22:44:51.575326 21995 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:51.575380 21995 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:51.575546 21995 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:44:51.576861 21995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/minicluster-data/master-0/data/instance:
uuid: "085a90a977d84d9e9dbd88d0e19d2f21"
format_stamp: "Formatted at 2026-08-06 22:44:51 on dist-test-slave-h3ft"
server_key: "cf8c3773b4e589f0cc77fb5d5fe04e53"
server_key_iv: "36c461530063889d5a5897f62d9aa24d"
server_key_version: "encryptionkey@0"
I20260806 22:44:51.577286 21995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/minicluster-data/master-0/wal/instance:
uuid: "085a90a977d84d9e9dbd88d0e19d2f21"
format_stamp: "Formatted at 2026-08-06 22:44:51 on dist-test-slave-h3ft"
server_key: "cf8c3773b4e589f0cc77fb5d5fe04e53"
server_key_iv: "36c461530063889d5a5897f62d9aa24d"
server_key_version: "encryptionkey@0"
I20260806 22:44:51.578876 21995 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:44:51.579766 22013 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:51.580057 21995 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:44:51.580170 21995 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/minicluster-data/master-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/minicluster-data/master-0/wal
uuid: "085a90a977d84d9e9dbd88d0e19d2f21"
format_stamp: "Formatted at 2026-08-06 22:44:51 on dist-test-slave-h3ft"
server_key: "cf8c3773b4e589f0cc77fb5d5fe04e53"
server_key_iv: "36c461530063889d5a5897f62d9aa24d"
server_key_version: "encryptionkey@0"
I20260806 22:44:51.580260 21995 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:51.599622 21995 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:51.602099 21995 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:51.602280 21995 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:51.607728 21995 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.254:41579
I20260806 22:44:51.607808 22089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.254:41579 every 8 connection(s)
I20260806 22:44:51.608245 21995 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
I20260806 22:44:51.609483 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 21995
I20260806 22:44:51.609611 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/minicluster-data/master-0/wal/instance
I20260806 22:44:51.609741 22090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:51.609863 18615 external_mini_cluster.cc:1428] Setting key e5a61d599ecfa3dae65dd17775ca6479
I20260806 22:44:51.614068 22090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 085a90a977d84d9e9dbd88d0e19d2f21: Bootstrap starting.
I20260806 22:44:51.614934 22090 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 085a90a977d84d9e9dbd88d0e19d2f21: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:51.615392 22090 log.cc:824] T 00000000000000000000000000000000 P 085a90a977d84d9e9dbd88d0e19d2f21: Log is configured to *not* fsync() on all Append() calls
I20260806 22:44:51.616729 22090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 085a90a977d84d9e9dbd88d0e19d2f21: No bootstrap required, opened a new log
Aug 06 22:44:51 dist-test-slave-h3ft krb5kdc[21976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056291, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:44:51.619388 22090 raft_consensus.cc:348] T 00000000000000000000000000000000 P 085a90a977d84d9e9dbd88d0e19d2f21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "085a90a977d84d9e9dbd88d0e19d2f21" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 41579 } }
I20260806 22:44:51.619633 22090 raft_consensus.cc:374] T 00000000000000000000000000000000 P 085a90a977d84d9e9dbd88d0e19d2f21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:51.619678 22090 raft_consensus.cc:729] T 00000000000000000000000000000000 P 085a90a977d84d9e9dbd88d0e19d2f21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 085a90a977d84d9e9dbd88d0e19d2f21, State: Initialized, Role: FOLLOWER
I20260806 22:44:51.619906 22090 consensus_queue.cc:260] T 00000000000000000000000000000000 P 085a90a977d84d9e9dbd88d0e19d2f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "085a90a977d84d9e9dbd88d0e19d2f21" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 41579 } }
I20260806 22:44:51.620033 22095 raft_consensus.cc:482] T 00000000000000000000000000000000 P 085a90a977d84d9e9dbd88d0e19d2f21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:44:51.620178 22095 raft_consensus.cc:504] T 00000000000000000000000000000000 P 085a90a977d84d9e9dbd88d0e19d2f21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "085a90a977d84d9e9dbd88d0e19d2f21" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 41579 } }
I20260806 22:44:51.620010 22090 raft_consensus.cc:388] T 00000000000000000000000000000000 P 085a90a977d84d9e9dbd88d0e19d2f21 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:44:51.620337 22095 leader_election.cc:302] T 00000000000000000000000000000000 P 085a90a977d84d9e9dbd88d0e19d2f21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 085a90a977d84d9e9dbd88d0e19d2f21; no voters:
I20260806 22:44:51.620383 22090 raft_consensus.cc:482] T 00000000000000000000000000000000 P 085a90a977d84d9e9dbd88d0e19d2f21 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:44:51.620445 22090 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 085a90a977d84d9e9dbd88d0e19d2f21 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:51.620606 22095 leader_election.cc:288] T 00000000000000000000000000000000 P 085a90a977d84d9e9dbd88d0e19d2f21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers
I20260806 22:44:51.621351 22090 raft_consensus.cc:504] T 00000000000000000000000000000000 P 085a90a977d84d9e9dbd88d0e19d2f21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "085a90a977d84d9e9dbd88d0e19d2f21" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 41579 } }
I20260806 22:44:51.621455 22090 leader_election.cc:302] T 00000000000000000000000000000000 P 085a90a977d84d9e9dbd88d0e19d2f21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 085a90a977d84d9e9dbd88d0e19d2f21; no voters:
I20260806 22:44:51.621500 22090 leader_election.cc:288] T 00000000000000000000000000000000 P 085a90a977d84d9e9dbd88d0e19d2f21 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:44:51.621523 22097 raft_consensus.cc:2741] T 00000000000000000000000000000000 P 085a90a977d84d9e9dbd88d0e19d2f21 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260806 22:44:51.621661 22097 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 085a90a977d84d9e9dbd88d0e19d2f21 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:44:51.621707 22097 raft_consensus.cc:686] T 00000000000000000000000000000000 P 085a90a977d84d9e9dbd88d0e19d2f21 [term 1 LEADER]: Becoming Leader. State: Replica: 085a90a977d84d9e9dbd88d0e19d2f21, State: Running, Role: LEADER
I20260806 22:44:51.621789 22090 sys_catalog.cc:564] T 00000000000000000000000000000000 P 085a90a977d84d9e9dbd88d0e19d2f21 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:44:51.621803 22097 consensus_queue.cc:237] T 00000000000000000000000000000000 P 085a90a977d84d9e9dbd88d0e19d2f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "085a90a977d84d9e9dbd88d0e19d2f21" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 41579 } }
I20260806 22:44:51.622574 22095 sys_catalog.cc:455] T 00000000000000000000000000000000 P 085a90a977d84d9e9dbd88d0e19d2f21 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "085a90a977d84d9e9dbd88d0e19d2f21" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "085a90a977d84d9e9dbd88d0e19d2f21" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 41579 } } }
I20260806 22:44:51.622754 22095 sys_catalog.cc:458] T 00000000000000000000000000000000 P 085a90a977d84d9e9dbd88d0e19d2f21 [sys.catalog]: This master's current role is: LEADER
I20260806 22:44:51.623042 22093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:51.610528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43616 (local address 127.18.45.254:41579)
0806 22:44:51.610736 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:51.610746 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:51.610776 (+ 30us) server_negotiation.cc:407] Connection header received
0806 22:44:51.611307 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:51.611316 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:51.611376 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:51.611528 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:51.612456 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:51.613495 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:51.614620 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:51.615174 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:51.615279 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:51.618263 (+ 2984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:51.618270 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:51.618270 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:51.618285 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:51.620569 (+ 2284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:51.620985 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:51.620989 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:51.620989 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:51.621125 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:51.621343 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:51.621344 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:51.621344 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:51.621753 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:51.622439 (+ 686us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:51.622540 (+ 101us) server_negotiation.cc:299] Negotiation successful
0806 22:44:51.622711 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":102,"threads_started":1}
I20260806 22:44:51.623389 22108 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:44:51.623028 22099 sys_catalog.cc:455] T 00000000000000000000000000000000 P 085a90a977d84d9e9dbd88d0e19d2f21 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 085a90a977d84d9e9dbd88d0e19d2f21. Latest consensus state: current_term: 1 leader_uuid: "085a90a977d84d9e9dbd88d0e19d2f21" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "085a90a977d84d9e9dbd88d0e19d2f21" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 41579 } } }
I20260806 22:44:51.623795 22099 sys_catalog.cc:458] T 00000000000000000000000000000000 P 085a90a977d84d9e9dbd88d0e19d2f21 [sys.catalog]: This master's current role is: LEADER
I20260806 22:44:51.624243 22108 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:44:51.626025 22108 catalog_manager.cc:1347] Generated new cluster ID: b61c99252d3b4e21a53c9e74fb0702c6
I20260806 22:44:51.626080 22108 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:44:51.640796 22108 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:44:51.641655 22108 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:44:51.650720 22108 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 085a90a977d84d9e9dbd88d0e19d2f21: Generated new TSK 0
I20260806 22:44:51.651070 22108 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.45.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab.
Entry for principal kudu/127.18.45.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab.
I20260806 22:44:51.686614 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.193:0
--local_ip_for_outbound_sockets=127.18.45.193
--webserver_interface=127.18.45.193
--webserver_port=0
--tserver_master_addrs=127.18.45.254:41579
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab
--principal=kudu/127.18.45.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:45837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:44:51.788166 22122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:51.788439 22122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:51.788466 22122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:51.789700 22122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:51.789742 22122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:51.789770 22122 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.193
I20260806 22:44:51.791394 22122 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:45837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab
--principal=kudu/127.18.45.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.45.193
--webserver_port=0
--tserver_master_addrs=127.18.45.254:41579
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22122
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.193
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:44:51.791652 22122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:51.791967 22122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:51.798245 22137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:51.798229 22134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:51.798226 22133 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:51.798561 22122 server_base.cc:1034] running on GCE node
I20260806 22:44:51.798791 22122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:51.799122 22122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:51.800309 22122 hybrid_clock.cc:648] HybridClock initialized: now 1786056291800288 us; error 43 us; skew 500 ppm
Aug 06 22:44:51 dist-test-slave-h3ft krb5kdc[21976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056291, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:51.803537 22122 init.cc:377] Logged in from keytab as kudu/127.18.45.193@KRBTEST.COM (short username kudu)
I20260806 22:44:51.804131 22122 webserver.cc:460] Webserver started at http://127.18.45.193:38349/ using document root <none> and password file <none>
I20260806 22:44:51.804442 22122 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:51.804525 22122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:51.804698 22122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:44:51.805999 22122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-0/data/instance:
uuid: "9c6115d5766d49d1a52aa9ec537e67d6"
format_stamp: "Formatted at 2026-08-06 22:44:51 on dist-test-slave-h3ft"
server_key: "c1b3a06862c81e85a922f197eec69eba"
server_key_iv: "397557c6d47578d1241e041ea07d5eec"
server_key_version: "encryptionkey@0"
I20260806 22:44:51.806432 22122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-0/wal/instance:
uuid: "9c6115d5766d49d1a52aa9ec537e67d6"
format_stamp: "Formatted at 2026-08-06 22:44:51 on dist-test-slave-h3ft"
server_key: "c1b3a06862c81e85a922f197eec69eba"
server_key_iv: "397557c6d47578d1241e041ea07d5eec"
server_key_version: "encryptionkey@0"
I20260806 22:44:51.808091 22122 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260806 22:44:51.809051 22146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:51.809262 22122 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:44:51.809391 22122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-0/wal
uuid: "9c6115d5766d49d1a52aa9ec537e67d6"
format_stamp: "Formatted at 2026-08-06 22:44:51 on dist-test-slave-h3ft"
server_key: "c1b3a06862c81e85a922f197eec69eba"
server_key_iv: "397557c6d47578d1241e041ea07d5eec"
server_key_version: "encryptionkey@0"
I20260806 22:44:51.809484 22122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:51.827980 22122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:51.830557 22122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:51.830772 22122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:51.831190 22122 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:44:51.831609 22122 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:44:51.831684 22122 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:51.831737 22122 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:44:51.831768 22122 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:51.839339 22122 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.193:34975
I20260806 22:44:51.839401 22314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.193:34975 every 8 connection(s)
I20260806 22:44:51.839804 22122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
I20260806 22:44:51.844635 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 22122
I20260806 22:44:51.844735 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-0/wal/instance
I20260806 22:44:51.845016 18615 external_mini_cluster.cc:1428] Setting key eb998a4248e234af8308dbbdc4ecb490
Aug 06 22:44:51 dist-test-slave-h3ft krb5kdc[21976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056291, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.193@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:44:51.852466 22093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:51.840773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:45625 (local address 127.18.45.254:41579)
0806 22:44:51.840887 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:51.840891 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:51.841130 (+ 239us) server_negotiation.cc:407] Connection header received
0806 22:44:51.841836 (+ 706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:51.841838 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:51.841855 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:51.841916 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:51.843265 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:51.843837 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:51.845500 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:51.845742 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:51.845826 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:51.848745 (+ 2919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:51.848746 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:51.848747 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:51.848756 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:51.850728 (+ 1972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:51.851251 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:51.851252 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:51.851253 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:51.851334 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:51.851602 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:51.851603 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:51.851603 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:51.851784 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:51.851855 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:51.852207 (+ 352us) server_negotiation.cc:299] Negotiation successful
0806 22:44:51.852329 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 22:44:51.852636 22318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:51.840726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:41579 (local address 127.18.45.193:45625)
0806 22:44:51.840955 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:51.841014 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:51.841683 (+ 669us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:51.842047 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:51.842054 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:51.842821 (+ 767us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:51.843149 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:51.843157 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:51.843954 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:51.843955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:51.845286 (+ 1331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:51.845290 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:51.845839 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:51.845840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:51.845991 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:51.846710 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:44:51.846720 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:51.848612 (+ 1892us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:51.850824 (+ 2212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:51.850832 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:51.850835 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:51.851165 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:51.851405 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:51.851405 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:51.851406 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:51.851524 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:51.851845 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:51.851850 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:51.851996 (+ 146us) client_negotiation.cc:769] Sending connection context
0806 22:44:51.852113 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 22:44:51.852277 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":102,"threads_started":1}
I20260806 22:44:51.853567 22316 heartbeater.cc:347] Connected to a master server at 127.18.45.254:41579
I20260806 22:44:51.853715 22316 heartbeater.cc:464] Registering TS with master...
I20260806 22:44:51.853960 22316 heartbeater.cc:511] Master 127.18.45.254:41579 requested a full tablet report, sending...
I20260806 22:44:51.854791 22040 ts_manager.cc:194] Registered new tserver with Master: 9c6115d5766d49d1a52aa9ec537e67d6 (127.18.45.193:34975)
I20260806 22:44:51.855923 22040 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.45.193@KRBTEST.COM'} at 127.18.45.193:45625
WARNING: no policy specified for kudu/127.18.45.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab.
Entry for principal kudu/127.18.45.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab.
I20260806 22:44:51.882844 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.194:0
--local_ip_for_outbound_sockets=127.18.45.194
--webserver_interface=127.18.45.194
--webserver_port=0
--tserver_master_addrs=127.18.45.254:41579
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab
--principal=kudu/127.18.45.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:45837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:44:51.981316 22322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:51.981621 22322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:51.981652 22322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:51.983174 22322 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:51.983225 22322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:51.983258 22322 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.194
I20260806 22:44:51.985008 22322 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:45837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab
--principal=kudu/127.18.45.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.45.194
--webserver_port=0
--tserver_master_addrs=127.18.45.254:41579
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22322
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.194
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:44:51.985312 22322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:51.985611 22322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:51.991808 22333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:51.991823 22335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:51.991823 22332 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:51.992213 22322 server_base.cc:1034] running on GCE node
I20260806 22:44:51.992456 22322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:51.992738 22322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:51.993953 22322 hybrid_clock.cc:648] HybridClock initialized: now 1786056291993919 us; error 55 us; skew 500 ppm
Aug 06 22:44:51 dist-test-slave-h3ft krb5kdc[21976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056291, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:51.997176 22322 init.cc:377] Logged in from keytab as kudu/127.18.45.194@KRBTEST.COM (short username kudu)
I20260806 22:44:51.997740 22322 webserver.cc:460] Webserver started at http://127.18.45.194:37667/ using document root <none> and password file <none>
I20260806 22:44:51.998018 22322 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:51.998091 22322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:51.998257 22322 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:44:51.999583 22322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-1/data/instance:
uuid: "1dff0fca0a89484cbed0a46ec86f61dd"
format_stamp: "Formatted at 2026-08-06 22:44:51 on dist-test-slave-h3ft"
server_key: "1547616e978a472aa98c74a48b743bbb"
server_key_iv: "d0a453669383178037be6c7659d63e2c"
server_key_version: "encryptionkey@0"
I20260806 22:44:52.000007 22322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-1/wal/instance:
uuid: "1dff0fca0a89484cbed0a46ec86f61dd"
format_stamp: "Formatted at 2026-08-06 22:44:51 on dist-test-slave-h3ft"
server_key: "1547616e978a472aa98c74a48b743bbb"
server_key_iv: "d0a453669383178037be6c7659d63e2c"
server_key_version: "encryptionkey@0"
I20260806 22:44:52.001606 22322 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:44:52.002553 22343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:52.002800 22322 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:44:52.002920 22322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-1/wal
uuid: "1dff0fca0a89484cbed0a46ec86f61dd"
format_stamp: "Formatted at 2026-08-06 22:44:51 on dist-test-slave-h3ft"
server_key: "1547616e978a472aa98c74a48b743bbb"
server_key_iv: "d0a453669383178037be6c7659d63e2c"
server_key_version: "encryptionkey@0"
I20260806 22:44:52.003078 22322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:52.037326 22322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:52.039734 22322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:52.039927 22322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:52.040261 22322 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:44:52.040663 22322 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:44:52.040719 22322 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:52.040764 22322 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:44:52.040793 22322 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:52.048199 22322 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.194:46081
I20260806 22:44:52.048254 22512 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.194:46081 every 8 connection(s)
I20260806 22:44:52.048655 22322 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
I20260806 22:44:52.051347 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 22322
I20260806 22:44:52.051488 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-1/wal/instance
I20260806 22:44:52.051717 18615 external_mini_cluster.cc:1428] Setting key 3f6d4b44bda06d0083a65e8ea15e1191
Aug 06 22:44:52 dist-test-slave-h3ft krb5kdc[21976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056291, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.194@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:44:52.060809 22093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:52.049573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:46963 (local address 127.18.45.254:41579)
0806 22:44:52.049650 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:52.049654 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:52.049990 (+ 336us) server_negotiation.cc:407] Connection header received
0806 22:44:52.050868 (+ 878us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:52.050870 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:52.050885 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:52.050951 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:52.052334 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:52.053028 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:52.054625 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:52.054835 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:52.054945 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:52.057582 (+ 2637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:52.057584 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:52.057585 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:52.057597 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:52.059239 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:52.059689 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:52.059690 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:52.059690 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:52.059762 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:52.060043 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:52.060044 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:52.060044 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:52.060181 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:52.060264 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:52.060583 (+ 319us) server_negotiation.cc:299] Negotiation successful
0806 22:44:52.060700 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 22:44:52.061075 22519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:52.049554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:41579 (local address 127.18.45.194:46963)
0806 22:44:52.049796 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:52.049853 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:52.050727 (+ 874us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:52.051091 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:52.051096 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:52.051926 (+ 830us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:52.052233 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:52.052240 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:52.053137 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:52.053138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:52.054517 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:52.054522 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:52.054976 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:52.054978 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:52.055197 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:52.055867 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:44:52.055877 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:52.057475 (+ 1598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:52.059337 (+ 1862us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:52.059346 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:52.059348 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:52.059606 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:52.059834 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:52.059835 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:52.059835 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:52.059966 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:52.060252 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:52.060260 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:52.060432 (+ 172us) client_negotiation.cc:769] Sending connection context
0806 22:44:52.060549 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 22:44:52.060735 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":98,"threads_started":1}
I20260806 22:44:52.061645 22513 heartbeater.cc:347] Connected to a master server at 127.18.45.254:41579
I20260806 22:44:52.061761 22513 heartbeater.cc:464] Registering TS with master...
I20260806 22:44:52.062064 22513 heartbeater.cc:511] Master 127.18.45.254:41579 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.18.45.195@KRBTEST.COM; defaulting to no policy
I20260806 22:44:52.062605 22040 ts_manager.cc:194] Registered new tserver with Master: 1dff0fca0a89484cbed0a46ec86f61dd (127.18.45.194:46081)
I20260806 22:44:52.063216 22040 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.45.194@KRBTEST.COM'} at 127.18.45.194:46963
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab.
Entry for principal kudu/127.18.45.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab.
I20260806 22:44:52.087410 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.195:0
--local_ip_for_outbound_sockets=127.18.45.195
--webserver_interface=127.18.45.195
--webserver_port=0
--tserver_master_addrs=127.18.45.254:41579
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab
--principal=kudu/127.18.45.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:45837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:44:52.188845 22525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:52.189095 22525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:52.189123 22525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:52.190497 22525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:52.190552 22525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:52.190582 22525 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.195
I20260806 22:44:52.192440 22525 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:45837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab
--principal=kudu/127.18.45.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.45.195
--webserver_port=0
--tserver_master_addrs=127.18.45.254:41579
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22525
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.195
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:44:52.192806 22525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:52.193166 22525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:52.199616 22531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:52.199630 22534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:52.199627 22532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:52.199898 22525 server_base.cc:1034] running on GCE node
I20260806 22:44:52.200174 22525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:52.200449 22525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:52.201614 22525 hybrid_clock.cc:648] HybridClock initialized: now 1786056292201600 us; error 31 us; skew 500 ppm
Aug 06 22:44:52 dist-test-slave-h3ft krb5kdc[21976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056292, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:52.204985 22525 init.cc:377] Logged in from keytab as kudu/127.18.45.195@KRBTEST.COM (short username kudu)
I20260806 22:44:52.205511 22525 webserver.cc:460] Webserver started at http://127.18.45.195:44173/ using document root <none> and password file <none>
I20260806 22:44:52.205773 22525 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:52.205821 22525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:52.205928 22525 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:44:52.207362 22525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-2/data/instance:
uuid: "37b6b9d10f3d4757afe4e0e43e62383c"
format_stamp: "Formatted at 2026-08-06 22:44:52 on dist-test-slave-h3ft"
server_key: "fceff5d320543e0a62d21f1b8e1aebc5"
server_key_iv: "a79c753b42df35c7e0efbacfc7eb3e9e"
server_key_version: "encryptionkey@0"
I20260806 22:44:52.207821 22525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-2/wal/instance:
uuid: "37b6b9d10f3d4757afe4e0e43e62383c"
format_stamp: "Formatted at 2026-08-06 22:44:52 on dist-test-slave-h3ft"
server_key: "fceff5d320543e0a62d21f1b8e1aebc5"
server_key_iv: "a79c753b42df35c7e0efbacfc7eb3e9e"
server_key_version: "encryptionkey@0"
I20260806 22:44:52.209362 22525 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260806 22:44:52.210217 22545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:52.210457 22525 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:44:52.210582 22525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-2/wal
uuid: "37b6b9d10f3d4757afe4e0e43e62383c"
format_stamp: "Formatted at 2026-08-06 22:44:52 on dist-test-slave-h3ft"
server_key: "fceff5d320543e0a62d21f1b8e1aebc5"
server_key_iv: "a79c753b42df35c7e0efbacfc7eb3e9e"
server_key_version: "encryptionkey@0"
I20260806 22:44:52.210671 22525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:52.224977 22525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:52.227754 22525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:52.227946 22525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:52.228255 22525 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:44:52.228719 22525 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:44:52.228780 22525 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:52.228830 22525 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:44:52.228859 22525 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:52.236263 22525 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.195:36553
I20260806 22:44:52.236320 22716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.195:36553 every 8 connection(s)
I20260806 22:44:52.236692 22525 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
Aug 06 22:44:52 dist-test-slave-h3ft krb5kdc[21976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056292, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.195@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:44:52.245745 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 22525
I20260806 22:44:52.245863 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-2/wal/instance
I20260806 22:44:52.246125 18615 external_mini_cluster.cc:1428] Setting key d6c5dff90a7e142048f83531a430c1ef
I20260806 22:44:52.247262 22093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:52.237627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.195:53583 (local address 127.18.45.254:41579)
0806 22:44:52.237711 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:52.237714 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:52.237972 (+ 258us) server_negotiation.cc:407] Connection header received
0806 22:44:52.238574 (+ 602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:52.238575 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:52.238588 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:52.238647 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:52.239801 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:52.240367 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:52.241425 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:52.241622 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:52.241675 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:52.244131 (+ 2456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:52.244132 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:52.244133 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:52.244140 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:52.245430 (+ 1290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:52.245993 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:52.245994 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:52.245995 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:52.246066 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:52.246381 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:52.246382 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:52.246382 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:52.246580 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:52.246650 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:52.247059 (+ 409us) server_negotiation.cc:299] Negotiation successful
0806 22:44:52.247154 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 22:44:52.247479 22721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:52.237599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:41579 (local address 127.18.45.195:53583)
0806 22:44:52.237820 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:52.237866 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:52.238469 (+ 603us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:52.238715 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:52.238719 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:52.239470 (+ 751us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:52.239711 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:52.239717 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:52.240461 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:52.240461 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:52.241349 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:52.241351 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:52.241699 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:52.241700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:52.241819 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:52.242394 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:44:52.242416 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:52.244034 (+ 1618us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:52.245538 (+ 1504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:52.245547 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:52.245550 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:52.245898 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:52.246135 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:52.246135 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:52.246136 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:52.246297 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:52.246652 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:52.246661 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:52.246849 (+ 188us) client_negotiation.cc:769] Sending connection context
0806 22:44:52.247012 (+ 163us) client_negotiation.cc:240] Negotiation successful
0806 22:44:52.247224 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":119,"threads_started":1}
I20260806 22:44:52.248051 22719 heartbeater.cc:347] Connected to a master server at 127.18.45.254:41579
I20260806 22:44:52.248145 22719 heartbeater.cc:464] Registering TS with master...
I20260806 22:44:52.248330 22719 heartbeater.cc:511] Master 127.18.45.254:41579 requested a full tablet report, sending...
I20260806 22:44:52.248732 22040 ts_manager.cc:194] Registered new tserver with Master: 37b6b9d10f3d4757afe4e0e43e62383c (127.18.45.195:36553)
I20260806 22:44:52.249161 22040 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.45.195@KRBTEST.COM'} at 127.18.45.195:53583
I20260806 22:44:52.258275 18615 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 22:44:52.275184 22093 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:44:52.273859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43620 (local address 127.18.45.254:41579)
0806 22:44:52.273999 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:52.274003 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:52.274118 (+ 115us) server_negotiation.cc:407] Connection header received
0806 22:44:52.274237 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:52.274239 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:52.274252 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:52.274317 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:52.275083 (+ 766us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43620: BlockingRecv error: recv got EOF from 127.0.0.1:43620 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
I20260806 22:44:52.276264 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 22122
I20260806 22:44:52.296697 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 22322
I20260806 22:44:52.312692 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 22525
I20260806 22:44:52.332952 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 21995
2026-08-06T22:44:52Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2301 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056275803658-18615-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:44:52 dist-test-slave-h3ft krb5kdc[22734](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:44:52 dist-test-slave-h3ft krb5kdc[22734](info): set up 2 sockets
Aug 06 22:44:52 dist-test-slave-h3ft krb5kdc[22734](info): commencing operation
krb5kdc: starting...
W20260806 22:44:53.616743 18615 mini_kdc.cc:121] Time spent starting KDC: real 1.252s user 0.006s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Aug 06 22:44:53 dist-test-slave-h3ft krb5kdc[22734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056293, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:44:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:44:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.45.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab.
Entry for principal kudu/127.18.45.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab.
I20260806 22:44:53.778785 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056275803658-18615-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056275803658-18615-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056275803658-18615-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:36573
--webserver_interface=127.18.45.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:45297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.45.254:36573
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:44:53.879761 22748 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:53.880097 22748 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:53.880146 22748 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:53.881596 22748 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:44:53.881677 22748 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:53.881703 22748 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:53.881738 22748 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:44:53.881765 22748 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:44:53.883713 22748 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:45297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056275803658-18615-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056275803658-18615-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.45.254:36573
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:36573
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.45.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22748
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056275803658-18615-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:44:53.884074 22748 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:53.884372 22748 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:53.890288 22762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:53.890300 22758 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:53.890493 22748 server_base.cc:1034] running on GCE node
W20260806 22:44:53.890604 22756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:53.890940 22748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:53.891345 22748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:53.892520 22748 hybrid_clock.cc:648] HybridClock initialized: now 1786056293892495 us; error 36 us; skew 500 ppm
Aug 06 22:44:53 dist-test-slave-h3ft krb5kdc[22734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056293, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:53.902194 22748 init.cc:377] Logged in from keytab as kudu/127.18.45.254@KRBTEST.COM (short username kudu)
I20260806 22:44:53.902870 22748 webserver.cc:460] Webserver started at http://127.18.45.254:38503/ using document root <none> and password file <none>
I20260806 22:44:53.903225 22748 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:53.903309 22748 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:53.903499 22748 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:44:53.904876 22748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056275803658-18615-0/minicluster-data/master-0/data/instance:
uuid: "da210b3bdf8a46168137ba73c7fa8461"
format_stamp: "Formatted at 2026-08-06 22:44:53 on dist-test-slave-h3ft"
server_key: "85137bcb027056b90552ee6e3ef453ca"
server_key_iv: "5dde258a5fb6ec5ed0437e9e986cd6b0"
server_key_version: "encryptionkey@0"
I20260806 22:44:53.905319 22748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056275803658-18615-0/minicluster-data/master-0/wal/instance:
uuid: "da210b3bdf8a46168137ba73c7fa8461"
format_stamp: "Formatted at 2026-08-06 22:44:53 on dist-test-slave-h3ft"
server_key: "85137bcb027056b90552ee6e3ef453ca"
server_key_iv: "5dde258a5fb6ec5ed0437e9e986cd6b0"
server_key_version: "encryptionkey@0"
I20260806 22:44:53.907246 22748 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260806 22:44:53.908375 22772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:53.908632 22748 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:44:53.908773 22748 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056275803658-18615-0/minicluster-data/master-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056275803658-18615-0/minicluster-data/master-0/wal
uuid: "da210b3bdf8a46168137ba73c7fa8461"
format_stamp: "Formatted at 2026-08-06 22:44:53 on dist-test-slave-h3ft"
server_key: "85137bcb027056b90552ee6e3ef453ca"
server_key_iv: "5dde258a5fb6ec5ed0437e9e986cd6b0"
server_key_version: "encryptionkey@0"
I20260806 22:44:53.908869 22748 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056275803658-18615-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056275803658-18615-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056275803658-18615-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:53.924633 22748 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:53.927548 22748 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:53.927762 22748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:53.933344 22748 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.254:36573
I20260806 22:44:53.933380 22849 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.254:36573 every 8 connection(s)
I20260806 22:44:53.934194 22748 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
I20260806 22:44:53.935228 22850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:53.937127 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 22748
I20260806 22:44:53.937371 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786056275803658-18615-0/minicluster-data/master-0/wal/instance
I20260806 22:44:53.937602 18615 external_mini_cluster.cc:1428] Setting key af3951e1285a7c932f78c44414de79e0
I20260806 22:44:53.939287 22850 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da210b3bdf8a46168137ba73c7fa8461: Bootstrap starting.
I20260806 22:44:53.940294 22850 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P da210b3bdf8a46168137ba73c7fa8461: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:53.940757 22850 log.cc:824] T 00000000000000000000000000000000 P da210b3bdf8a46168137ba73c7fa8461: Log is configured to *not* fsync() on all Append() calls
I20260806 22:44:53.941639 22850 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da210b3bdf8a46168137ba73c7fa8461: No bootstrap required, opened a new log
I20260806 22:44:53.944622 22850 raft_consensus.cc:348] T 00000000000000000000000000000000 P da210b3bdf8a46168137ba73c7fa8461 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da210b3bdf8a46168137ba73c7fa8461" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 36573 } }
I20260806 22:44:53.944865 22850 raft_consensus.cc:374] T 00000000000000000000000000000000 P da210b3bdf8a46168137ba73c7fa8461 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:53.944913 22850 raft_consensus.cc:729] T 00000000000000000000000000000000 P da210b3bdf8a46168137ba73c7fa8461 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da210b3bdf8a46168137ba73c7fa8461, State: Initialized, Role: FOLLOWER
I20260806 22:44:53.945094 22850 consensus_queue.cc:260] T 00000000000000000000000000000000 P da210b3bdf8a46168137ba73c7fa8461 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da210b3bdf8a46168137ba73c7fa8461" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 36573 } }
I20260806 22:44:53.945204 22850 raft_consensus.cc:388] T 00000000000000000000000000000000 P da210b3bdf8a46168137ba73c7fa8461 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:44:53.945248 22850 raft_consensus.cc:482] T 00000000000000000000000000000000 P da210b3bdf8a46168137ba73c7fa8461 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:44:53.945317 22850 raft_consensus.cc:3037] T 00000000000000000000000000000000 P da210b3bdf8a46168137ba73c7fa8461 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:53.946198 22850 raft_consensus.cc:504] T 00000000000000000000000000000000 P da210b3bdf8a46168137ba73c7fa8461 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da210b3bdf8a46168137ba73c7fa8461" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 36573 } }
I20260806 22:44:53.946322 22850 leader_election.cc:302] T 00000000000000000000000000000000 P da210b3bdf8a46168137ba73c7fa8461 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: da210b3bdf8a46168137ba73c7fa8461; no voters:
Aug 06 22:44:53 dist-test-slave-h3ft krb5kdc[22734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056293, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:44:53.946573 22850 leader_election.cc:288] T 00000000000000000000000000000000 P da210b3bdf8a46168137ba73c7fa8461 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:44:53.946610 22856 raft_consensus.cc:2781] T 00000000000000000000000000000000 P da210b3bdf8a46168137ba73c7fa8461 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:44:53.946754 22856 raft_consensus.cc:686] T 00000000000000000000000000000000 P da210b3bdf8a46168137ba73c7fa8461 [term 1 LEADER]: Becoming Leader. State: Replica: da210b3bdf8a46168137ba73c7fa8461, State: Running, Role: LEADER
I20260806 22:44:53.946964 22850 sys_catalog.cc:564] T 00000000000000000000000000000000 P da210b3bdf8a46168137ba73c7fa8461 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:44:53.946938 22856 consensus_queue.cc:237] T 00000000000000000000000000000000 P da210b3bdf8a46168137ba73c7fa8461 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da210b3bdf8a46168137ba73c7fa8461" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 36573 } }
I20260806 22:44:53.947695 22857 sys_catalog.cc:455] T 00000000000000000000000000000000 P da210b3bdf8a46168137ba73c7fa8461 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "da210b3bdf8a46168137ba73c7fa8461" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da210b3bdf8a46168137ba73c7fa8461" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 36573 } } }
I20260806 22:44:53.948213 22857 sys_catalog.cc:458] T 00000000000000000000000000000000 P da210b3bdf8a46168137ba73c7fa8461 [sys.catalog]: This master's current role is: LEADER
I20260806 22:44:53.948448 22858 sys_catalog.cc:455] T 00000000000000000000000000000000 P da210b3bdf8a46168137ba73c7fa8461 [sys.catalog]: SysCatalogTable state changed. Reason: New leader da210b3bdf8a46168137ba73c7fa8461. Latest consensus state: current_term: 1 leader_uuid: "da210b3bdf8a46168137ba73c7fa8461" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da210b3bdf8a46168137ba73c7fa8461" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 36573 } } }
I20260806 22:44:53.948575 22858 sys_catalog.cc:458] T 00000000000000000000000000000000 P da210b3bdf8a46168137ba73c7fa8461 [sys.catalog]: This master's current role is: LEADER
W20260806 22:44:53.949759 22879 catalog_manager.cc:1558] T 00000000000000000000000000000000 P da210b3bdf8a46168137ba73c7fa8461: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:44:53.949829 22879 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:44:53.949903 22880 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:44:53.950134 22880 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:44:53.952555 22854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:53.938431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41680 (local address 127.18.45.254:36573)
0806 22:44:53.938683 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:53.938693 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:53.938724 (+ 31us) server_negotiation.cc:407] Connection header received
0806 22:44:53.939328 (+ 604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:53.939337 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:53.939422 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:53.939589 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:53.940677 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:53.941797 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:53.943090 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:53.943606 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:53.943712 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:53.947346 (+ 3634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:53.947353 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:53.947353 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:53.947373 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:53.950156 (+ 2783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:53.950797 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:53.950802 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:53.950802 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:53.950969 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:53.951331 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:53.951332 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:53.951332 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:53.951705 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:53.951886 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:53.952104 (+ 218us) server_negotiation.cc:299] Negotiation successful
0806 22:44:53.952308 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":132,"threads_started":1}
I20260806 22:44:53.953128 22880 catalog_manager.cc:1347] Generated new cluster ID: 8353388629444e9cb78d8273d8b72a63
I20260806 22:44:53.953192 22880 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:44:53.955816 18615 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:44:53.967518 22880 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:44:53.968413 22880 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:44:53.977169 22854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:53.972703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41688 (local address 127.18.45.254:36573)
0806 22:44:53.972771 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:53.972773 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:53.972958 (+ 185us) server_negotiation.cc:407] Connection header received
0806 22:44:53.973055 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:53.973057 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:53.973072 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:53.973135 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:53.973653 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:53.974279 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:53.975270 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:53.975544 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:53.975649 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:53.976564 (+ 915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:53.976566 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:53.976566 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:44:53.976817 (+ 251us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:53.976848 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:44:53.976855 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:53.976935 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:53.977012 (+ 77us) server_negotiation.cc:299] Negotiation successful
0806 22:44:53.977049 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 22:44:53.978610 22880 catalog_manager.cc:5868] T 00000000000000000000000000000000 P da210b3bdf8a46168137ba73c7fa8461: Generated new TSK 0
I20260806 22:44:53.978834 22880 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:44:53.984999 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 22748
2026-08-06T22:44:53Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1649 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:44:54 dist-test-slave-h3ft krb5kdc[22894](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:44:54 dist-test-slave-h3ft krb5kdc[22894](info): set up 2 sockets
Aug 06 22:44:54 dist-test-slave-h3ft krb5kdc[22894](info): commencing operation
krb5kdc: starting...
W20260806 22:44:55.303078 18615 mini_kdc.cc:121] Time spent starting KDC: real 1.289s user 0.003s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Aug 06 22:44:55 dist-test-slave-h3ft krb5kdc[22894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056295, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:44:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:44:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.45.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab.
Entry for principal kudu/127.18.45.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab.
I20260806 22:44:55.452687 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:45131
--webserver_interface=127.18.45.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:34933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.45.254:45131
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:44:55.548755 22916 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:55.549089 22916 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:55.549135 22916 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:55.550469 22916 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:44:55.550554 22916 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:55.550581 22916 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:55.550604 22916 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:44:55.550647 22916 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:44:55.552438 22916 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:34933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.45.254:45131
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:45131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.45.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22916
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:44:55.552812 22916 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:55.553148 22916 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:55.559007 22928 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:55.558990 22924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:55.559077 22931 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:55.559489 22916 server_base.cc:1034] running on GCE node
I20260806 22:44:55.559808 22916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:55.560175 22916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:55.561349 22916 hybrid_clock.cc:648] HybridClock initialized: now 1786056295561337 us; error 53 us; skew 500 ppm
Aug 06 22:44:55 dist-test-slave-h3ft krb5kdc[22894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056295, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:55.564538 22916 init.cc:377] Logged in from keytab as kudu/127.18.45.254@KRBTEST.COM (short username kudu)
I20260806 22:44:55.565124 22916 webserver.cc:460] Webserver started at http://127.18.45.254:41315/ using document root <none> and password file <none>
I20260806 22:44:55.565419 22916 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:55.565502 22916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:55.565673 22916 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:44:55.566962 22916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/minicluster-data/master-0/data/instance:
uuid: "0303f223a4634a10883a642304a694cf"
format_stamp: "Formatted at 2026-08-06 22:44:55 on dist-test-slave-h3ft"
server_key: "2d64c0172882763e4e2533adac631586"
server_key_iv: "a898326d47d5c503ef33b0c3cde7f3d4"
server_key_version: "encryptionkey@0"
I20260806 22:44:55.567464 22916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/minicluster-data/master-0/wal/instance:
uuid: "0303f223a4634a10883a642304a694cf"
format_stamp: "Formatted at 2026-08-06 22:44:55 on dist-test-slave-h3ft"
server_key: "2d64c0172882763e4e2533adac631586"
server_key_iv: "a898326d47d5c503ef33b0c3cde7f3d4"
server_key_version: "encryptionkey@0"
I20260806 22:44:55.569058 22916 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 22:44:55.570032 22941 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:55.570278 22916 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:44:55.570415 22916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/minicluster-data/master-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/minicluster-data/master-0/wal
uuid: "0303f223a4634a10883a642304a694cf"
format_stamp: "Formatted at 2026-08-06 22:44:55 on dist-test-slave-h3ft"
server_key: "2d64c0172882763e4e2533adac631586"
server_key_iv: "a898326d47d5c503ef33b0c3cde7f3d4"
server_key_version: "encryptionkey@0"
I20260806 22:44:55.570508 22916 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:55.591747 22916 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:55.594599 22916 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:55.594834 22916 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:55.600634 22916 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.254:45131
I20260806 22:44:55.600682 23016 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.254:45131 every 8 connection(s)
I20260806 22:44:55.601155 22916 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
I20260806 22:44:55.602739 23017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:55.605934 23017 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0303f223a4634a10883a642304a694cf: Bootstrap starting.
I20260806 22:44:55.606742 23017 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0303f223a4634a10883a642304a694cf: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:55.607097 23017 log.cc:824] T 00000000000000000000000000000000 P 0303f223a4634a10883a642304a694cf: Log is configured to *not* fsync() on all Append() calls
I20260806 22:44:55.608112 23017 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0303f223a4634a10883a642304a694cf: No bootstrap required, opened a new log
I20260806 22:44:55.610563 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 22916
I20260806 22:44:55.610440 23017 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0303f223a4634a10883a642304a694cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0303f223a4634a10883a642304a694cf" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 45131 } }
I20260806 22:44:55.610677 23017 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0303f223a4634a10883a642304a694cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:55.610705 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/minicluster-data/master-0/wal/instance
I20260806 22:44:55.610720 23017 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0303f223a4634a10883a642304a694cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0303f223a4634a10883a642304a694cf, State: Initialized, Role: FOLLOWER
I20260806 22:44:55.610980 18615 external_mini_cluster.cc:1428] Setting key 074eea3d02a85c14640f198786493fac
I20260806 22:44:55.610965 23017 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0303f223a4634a10883a642304a694cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0303f223a4634a10883a642304a694cf" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 45131 } }
I20260806 22:44:55.611119 23017 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0303f223a4634a10883a642304a694cf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:44:55.611171 23017 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0303f223a4634a10883a642304a694cf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:44:55.611235 23017 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0303f223a4634a10883a642304a694cf [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:55.612808 23017 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0303f223a4634a10883a642304a694cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0303f223a4634a10883a642304a694cf" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 45131 } }
I20260806 22:44:55.612975 23017 leader_election.cc:302] T 00000000000000000000000000000000 P 0303f223a4634a10883a642304a694cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0303f223a4634a10883a642304a694cf; no voters:
I20260806 22:44:55.613188 23017 leader_election.cc:288] T 00000000000000000000000000000000 P 0303f223a4634a10883a642304a694cf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:44:55.613303 23024 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0303f223a4634a10883a642304a694cf [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:44:55.613540 23017 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0303f223a4634a10883a642304a694cf [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:44:55.613557 23024 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0303f223a4634a10883a642304a694cf [term 1 LEADER]: Becoming Leader. State: Replica: 0303f223a4634a10883a642304a694cf, State: Running, Role: LEADER
I20260806 22:44:55.613858 23024 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0303f223a4634a10883a642304a694cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0303f223a4634a10883a642304a694cf" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 45131 } }
I20260806 22:44:55.614497 23026 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0303f223a4634a10883a642304a694cf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0303f223a4634a10883a642304a694cf. Latest consensus state: current_term: 1 leader_uuid: "0303f223a4634a10883a642304a694cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0303f223a4634a10883a642304a694cf" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 45131 } } }
I20260806 22:44:55.614602 23026 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0303f223a4634a10883a642304a694cf [sys.catalog]: This master's current role is: LEADER
I20260806 22:44:55.614671 23025 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0303f223a4634a10883a642304a694cf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0303f223a4634a10883a642304a694cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0303f223a4634a10883a642304a694cf" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 45131 } } }
I20260806 22:44:55.614748 23025 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0303f223a4634a10883a642304a694cf [sys.catalog]: This master's current role is: LEADER
I20260806 22:44:55.615219 23040 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:44:55.616307 23040 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:44:55.618533 23040 catalog_manager.cc:1347] Generated new cluster ID: d54e3413241045e8ad17a732e933fc01
I20260806 22:44:55.618598 23040 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:44:55 dist-test-slave-h3ft krb5kdc[22894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056295, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:44:55.624435 23023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:55.611706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57240 (local address 127.18.45.254:45131)
0806 22:44:55.611939 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:55.611950 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:55.611988 (+ 38us) server_negotiation.cc:407] Connection header received
0806 22:44:55.612120 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:55.612126 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:55.612182 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:55.612325 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:55.613433 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:55.615328 (+ 1895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:55.616436 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:55.617183 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:55.617306 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:55.620522 (+ 3216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:55.620526 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:55.620527 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:55.620542 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:55.622520 (+ 1978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:55.622975 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:55.622979 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:55.622980 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:55.623163 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:55.623406 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:55.623407 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:55.623408 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:55.623718 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:55.623925 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:55.624005 (+ 80us) server_negotiation.cc:299] Negotiation successful
0806 22:44:55.624166 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":7,"server-negotiator.queue_time_us":162,"thread_start_us":124,"threads_started":1}
I20260806 22:44:55.640452 23040 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:44:55.641319 23040 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:44:55.651539 23040 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0303f223a4634a10883a642304a694cf: Generated new TSK 0
I20260806 22:44:55.651862 23040 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.45.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab.
Entry for principal kudu/127.18.45.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab.
I20260806 22:44:55.691661 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.193:0
--local_ip_for_outbound_sockets=127.18.45.193
--webserver_interface=127.18.45.193
--webserver_port=0
--tserver_master_addrs=127.18.45.254:45131
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab
--principal=kudu/127.18.45.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:34933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:44:55.795475 23057 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:55.795799 23057 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:55.795846 23057 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:55.797209 23057 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:55.797286 23057 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:55.797333 23057 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.193
I20260806 22:44:55.799075 23057 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:34933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab
--principal=kudu/127.18.45.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.45.193
--webserver_port=0
--tserver_master_addrs=127.18.45.254:45131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23057
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.193
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:44:55.799422 23057 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:55.799727 23057 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:55.806080 23064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:55.806080 23067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:55.806073 23065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:55.806568 23057 server_base.cc:1034] running on GCE node
I20260806 22:44:55.806833 23057 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:55.807233 23057 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:55.808401 23057 hybrid_clock.cc:648] HybridClock initialized: now 1786056295808376 us; error 53 us; skew 500 ppm
Aug 06 22:44:55 dist-test-slave-h3ft krb5kdc[22894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056295, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:55.811851 23057 init.cc:377] Logged in from keytab as kudu/127.18.45.193@KRBTEST.COM (short username kudu)
I20260806 22:44:55.812479 23057 webserver.cc:460] Webserver started at http://127.18.45.193:37335/ using document root <none> and password file <none>
I20260806 22:44:55.812798 23057 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:55.812877 23057 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:55.813171 23057 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:44:55.814695 23057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-0/data/instance:
uuid: "f36dda06e35d418891038c35497c4d25"
format_stamp: "Formatted at 2026-08-06 22:44:55 on dist-test-slave-h3ft"
server_key: "3a647bef463555ac6e2698d836cab604"
server_key_iv: "f24e3606e3db04996e249f8326617537"
server_key_version: "encryptionkey@0"
I20260806 22:44:55.815218 23057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-0/wal/instance:
uuid: "f36dda06e35d418891038c35497c4d25"
format_stamp: "Formatted at 2026-08-06 22:44:55 on dist-test-slave-h3ft"
server_key: "3a647bef463555ac6e2698d836cab604"
server_key_iv: "f24e3606e3db04996e249f8326617537"
server_key_version: "encryptionkey@0"
I20260806 22:44:55.816890 23057 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:44:55.817947 23081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:55.818207 23057 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:44:55.818351 23057 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-0/wal
uuid: "f36dda06e35d418891038c35497c4d25"
format_stamp: "Formatted at 2026-08-06 22:44:55 on dist-test-slave-h3ft"
server_key: "3a647bef463555ac6e2698d836cab604"
server_key_iv: "f24e3606e3db04996e249f8326617537"
server_key_version: "encryptionkey@0"
I20260806 22:44:55.818440 23057 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:55.854950 23057 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:55.857450 23057 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:55.857656 23057 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:55.857990 23057 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:44:55.858403 23057 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:44:55.858467 23057 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:55.858515 23057 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:44:55.858543 23057 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:55.865959 23057 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.193:41951
I20260806 22:44:55.866011 23258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.193:41951 every 8 connection(s)
I20260806 22:44:55.866401 23057 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
I20260806 22:44:55.870863 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 23057
I20260806 22:44:55.871040 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-0/wal/instance
I20260806 22:44:55.871290 18615 external_mini_cluster.cc:1428] Setting key 104e51c56c1f7f86440cb2f21ce09c2e
Aug 06 22:44:55 dist-test-slave-h3ft krb5kdc[22894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056295, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.193@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:44:55.879139 23023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:55.867364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:49473 (local address 127.18.45.254:45131)
0806 22:44:55.867441 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:55.867444 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:55.867681 (+ 237us) server_negotiation.cc:407] Connection header received
0806 22:44:55.868323 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:55.868324 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:55.868343 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:55.868421 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:55.869617 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:55.870229 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:55.871809 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:55.872143 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:55.872226 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:55.875612 (+ 3386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:55.875615 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:55.875616 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:55.875627 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:55.877442 (+ 1815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:55.877919 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:55.877921 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:55.877921 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:55.878004 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:55.878291 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:55.878292 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:55.878293 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:55.878478 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:55.878575 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:55.878873 (+ 298us) server_negotiation.cc:299] Negotiation successful
0806 22:44:55.878984 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 22:44:55.879333 23261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:55.867306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:45131 (local address 127.18.45.193:49473)
0806 22:44:55.867516 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:55.867568 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:55.868217 (+ 649us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:55.868513 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:55.868517 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:55.869249 (+ 732us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:55.869505 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:55.869512 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:55.870321 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:55.870322 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:55.871710 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:55.871714 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:55.872257 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:55.872259 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:55.872438 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:55.873185 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:44:55.873199 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:55.875496 (+ 2297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:55.877562 (+ 2066us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:55.877570 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:55.877573 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:55.877841 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:55.878093 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:55.878093 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:55.878094 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:55.878219 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:55.878564 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:55.878571 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:55.878719 (+ 148us) client_negotiation.cc:769] Sending connection context
0806 22:44:55.878819 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 22:44:55.878972 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":116,"threads_started":1}
I20260806 22:44:55.880170 23259 heartbeater.cc:347] Connected to a master server at 127.18.45.254:45131
I20260806 22:44:55.880307 23259 heartbeater.cc:464] Registering TS with master...
I20260806 22:44:55.880559 23259 heartbeater.cc:511] Master 127.18.45.254:45131 requested a full tablet report, sending...
I20260806 22:44:55.881292 22964 ts_manager.cc:194] Registered new tserver with Master: f36dda06e35d418891038c35497c4d25 (127.18.45.193:41951)
I20260806 22:44:55.882066 22964 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.45.193@KRBTEST.COM'} at 127.18.45.193:49473
WARNING: no policy specified for kudu/127.18.45.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab.
Entry for principal kudu/127.18.45.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab.
I20260806 22:44:55.907546 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.194:0
--local_ip_for_outbound_sockets=127.18.45.194
--webserver_interface=127.18.45.194
--webserver_port=0
--tserver_master_addrs=127.18.45.254:45131
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab
--principal=kudu/127.18.45.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:34933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:44:56.005266 23265 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:56.005565 23265 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:56.005606 23265 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:56.006870 23265 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:56.006943 23265 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:56.007032 23265 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.194
I20260806 22:44:56.008677 23265 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:34933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab
--principal=kudu/127.18.45.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.45.194
--webserver_port=0
--tserver_master_addrs=127.18.45.254:45131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23265
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.194
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:44:56.008987 23265 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:56.009265 23265 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:56.016253 23281 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:56.016248 23273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:56.016583 23265 server_base.cc:1034] running on GCE node
W20260806 22:44:56.016398 23272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:56.016960 23265 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:56.017272 23265 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:56.018460 23265 hybrid_clock.cc:648] HybridClock initialized: now 1786056296018440 us; error 43 us; skew 500 ppm
Aug 06 22:44:56 dist-test-slave-h3ft krb5kdc[22894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056296, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:56.021711 23265 init.cc:377] Logged in from keytab as kudu/127.18.45.194@KRBTEST.COM (short username kudu)
I20260806 22:44:56.022284 23265 webserver.cc:460] Webserver started at http://127.18.45.194:37301/ using document root <none> and password file <none>
I20260806 22:44:56.022559 23265 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:56.022610 23265 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:56.022794 23265 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:44:56.024106 23265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-1/data/instance:
uuid: "f97438856ba140b2b2d67055635a5eb1"
format_stamp: "Formatted at 2026-08-06 22:44:56 on dist-test-slave-h3ft"
server_key: "aed02b1e5d418b3685393453fcd7b4a5"
server_key_iv: "ca63e89ca218db084a6c87498a77302b"
server_key_version: "encryptionkey@0"
I20260806 22:44:56.024549 23265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-1/wal/instance:
uuid: "f97438856ba140b2b2d67055635a5eb1"
format_stamp: "Formatted at 2026-08-06 22:44:56 on dist-test-slave-h3ft"
server_key: "aed02b1e5d418b3685393453fcd7b4a5"
server_key_iv: "ca63e89ca218db084a6c87498a77302b"
server_key_version: "encryptionkey@0"
I20260806 22:44:56.026144 23265 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260806 22:44:56.026994 23291 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:56.027274 23265 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:44:56.027403 23265 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-1/wal
uuid: "f97438856ba140b2b2d67055635a5eb1"
format_stamp: "Formatted at 2026-08-06 22:44:56 on dist-test-slave-h3ft"
server_key: "aed02b1e5d418b3685393453fcd7b4a5"
server_key_iv: "ca63e89ca218db084a6c87498a77302b"
server_key_version: "encryptionkey@0"
I20260806 22:44:56.027482 23265 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:56.045392 23265 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:56.048148 23265 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:56.048393 23265 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:56.048727 23265 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:44:56.049122 23265 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:44:56.049160 23265 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:56.049188 23265 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:44:56.049201 23265 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:56.057045 23265 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.194:41387
I20260806 22:44:56.057092 23459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.194:41387 every 8 connection(s)
I20260806 22:44:56.057562 23265 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
I20260806 22:44:56.065572 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 23265
I20260806 22:44:56.065729 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-1/wal/instance
Aug 06 22:44:56 dist-test-slave-h3ft krb5kdc[22894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056296, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.194@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:44:56.066022 18615 external_mini_cluster.cc:1428] Setting key 84fa0134776ba11caf131e79d6fd9e8f
I20260806 22:44:56.070202 23023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:56.058680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:38405 (local address 127.18.45.254:45131)
0806 22:44:56.058764 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:56.058767 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:56.059041 (+ 274us) server_negotiation.cc:407] Connection header received
0806 22:44:56.059715 (+ 674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:56.059716 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:56.059730 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:56.059799 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:56.061064 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:56.061653 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:56.062837 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:56.063175 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:56.063263 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:56.066546 (+ 3283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:56.066548 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:56.066548 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:56.066559 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:56.068428 (+ 1869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:56.068975 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:56.068977 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:56.068978 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:56.069056 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:56.069379 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:56.069380 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:56.069380 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:56.069531 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:56.069626 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:56.069970 (+ 344us) server_negotiation.cc:299] Negotiation successful
0806 22:44:56.070090 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 22:44:56.070479 23465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:56.058625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:45131 (local address 127.18.45.194:38405)
0806 22:44:56.058832 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:56.058896 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:56.059603 (+ 707us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:56.059931 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:56.059936 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:56.060631 (+ 695us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:56.060895 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:56.060901 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:56.061765 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:56.061766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:56.062727 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:56.062731 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:56.063297 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:56.063304 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:56.063519 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:56.064275 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:44:56.064286 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:56.066392 (+ 2106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:56.068550 (+ 2158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:56.068562 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:56.068565 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:56.068865 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:56.069138 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:56.069139 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:56.069140 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:56.069291 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:56.069610 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:56.069619 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:56.069802 (+ 183us) client_negotiation.cc:769] Sending connection context
0806 22:44:56.069931 (+ 129us) client_negotiation.cc:240] Negotiation successful
0806 22:44:56.070138 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":111,"threads_started":1}
I20260806 22:44:56.071173 23460 heartbeater.cc:347] Connected to a master server at 127.18.45.254:45131
I20260806 22:44:56.071313 23460 heartbeater.cc:464] Registering TS with master...
I20260806 22:44:56.071539 23460 heartbeater.cc:511] Master 127.18.45.254:45131 requested a full tablet report, sending...
I20260806 22:44:56.072081 22964 ts_manager.cc:194] Registered new tserver with Master: f97438856ba140b2b2d67055635a5eb1 (127.18.45.194:41387)
I20260806 22:44:56.072535 22964 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.45.194@KRBTEST.COM'} at 127.18.45.194:38405
WARNING: no policy specified for kudu/127.18.45.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab.
Entry for principal kudu/127.18.45.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab.
I20260806 22:44:56.104130 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.195:0
--local_ip_for_outbound_sockets=127.18.45.195
--webserver_interface=127.18.45.195
--webserver_port=0
--tserver_master_addrs=127.18.45.254:45131
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab
--principal=kudu/127.18.45.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:34933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:44:56.203915 23468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:56.204242 23468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:56.204294 23468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:56.205730 23468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:56.205824 23468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:56.205878 23468 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.195
I20260806 22:44:56.207684 23468 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:34933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab
--principal=kudu/127.18.45.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.45.195
--webserver_port=0
--tserver_master_addrs=127.18.45.254:45131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23468
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.195
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:44:56.208048 23468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:56.208343 23468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:56.214486 23475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:56.214486 23479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:56.214576 23474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:56.215063 23468 server_base.cc:1034] running on GCE node
I20260806 22:44:56.215349 23468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:56.215667 23468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:56.216821 23468 hybrid_clock.cc:648] HybridClock initialized: now 1786056296216802 us; error 45 us; skew 500 ppm
Aug 06 22:44:56 dist-test-slave-h3ft krb5kdc[22894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056296, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:56.219887 23468 init.cc:377] Logged in from keytab as kudu/127.18.45.195@KRBTEST.COM (short username kudu)
I20260806 22:44:56.220489 23468 webserver.cc:460] Webserver started at http://127.18.45.195:41693/ using document root <none> and password file <none>
I20260806 22:44:56.220789 23468 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:56.220866 23468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:56.221066 23468 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:44:56.222491 23468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-2/data/instance:
uuid: "8b4e88ab177c438ab6cf76d4fe99b8eb"
format_stamp: "Formatted at 2026-08-06 22:44:56 on dist-test-slave-h3ft"
server_key: "c3f8a15641e53ceec8ee2ada9190c407"
server_key_iv: "4df22e63d0a1211f18692ba7683e5b1c"
server_key_version: "encryptionkey@0"
I20260806 22:44:56.222962 23468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-2/wal/instance:
uuid: "8b4e88ab177c438ab6cf76d4fe99b8eb"
format_stamp: "Formatted at 2026-08-06 22:44:56 on dist-test-slave-h3ft"
server_key: "c3f8a15641e53ceec8ee2ada9190c407"
server_key_iv: "4df22e63d0a1211f18692ba7683e5b1c"
server_key_version: "encryptionkey@0"
I20260806 22:44:56.224577 23468 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:44:56.225476 23488 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:56.225726 23468 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:44:56.225848 23468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-2/wal
uuid: "8b4e88ab177c438ab6cf76d4fe99b8eb"
format_stamp: "Formatted at 2026-08-06 22:44:56 on dist-test-slave-h3ft"
server_key: "c3f8a15641e53ceec8ee2ada9190c407"
server_key_iv: "4df22e63d0a1211f18692ba7683e5b1c"
server_key_version: "encryptionkey@0"
I20260806 22:44:56.225929 23468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:56.236855 23468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:56.239347 23468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:56.239532 23468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:56.239833 23468 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:44:56.240221 23468 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:44:56.240279 23468 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:56.240331 23468 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:44:56.240366 23468 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:56.248026 23468 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.195:34295
I20260806 22:44:56.248088 23648 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.195:34295 every 8 connection(s)
I20260806 22:44:56.248514 23468 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
I20260806 22:44:56.251927 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 23468
I20260806 22:44:56.252089 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786056275803658-18615-0/minicluster-data/ts-2/wal/instance
I20260806 22:44:56.252298 18615 external_mini_cluster.cc:1428] Setting key e9d28b7c6bcf16c4e2c400f0bbbaee2d
Aug 06 22:44:56 dist-test-slave-h3ft krb5kdc[22894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056296, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.195@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:44:56.259912 23023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:56.249490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.195:56559 (local address 127.18.45.254:45131)
0806 22:44:56.249595 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:56.249599 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:56.249851 (+ 252us) server_negotiation.cc:407] Connection header received
0806 22:44:56.250534 (+ 683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:56.250535 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:56.250553 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:56.250621 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:56.251884 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:56.252584 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:56.254044 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:56.254249 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:56.254309 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:56.256863 (+ 2554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:56.256865 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:56.256866 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:56.256874 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:56.258459 (+ 1585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:56.258890 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:56.258892 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:56.258892 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:56.258952 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:56.259276 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:56.259277 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:56.259277 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:56.259411 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:56.259472 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:56.259757 (+ 285us) server_negotiation.cc:299] Negotiation successful
0806 22:44:56.259829 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 22:44:56.260195 23653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:56.249468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:45131 (local address 127.18.45.195:56559)
0806 22:44:56.249677 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:56.249724 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:56.250395 (+ 671us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:56.250714 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:56.250719 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:56.251481 (+ 762us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:56.251750 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:56.251756 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:56.252726 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:56.252728 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:56.253942 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:56.253946 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:56.254335 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:56.254335 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:56.254451 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:56.255139 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:44:56.255149 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:56.256760 (+ 1611us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:56.258544 (+ 1784us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:56.258553 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:56.258556 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:56.258821 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:56.259091 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:56.259092 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:56.259092 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:56.259213 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:56.259474 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:56.259480 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:56.259627 (+ 147us) client_negotiation.cc:769] Sending connection context
0806 22:44:56.259714 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 22:44:56.259866 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":85,"threads_started":1}
I20260806 22:44:56.260885 23650 heartbeater.cc:347] Connected to a master server at 127.18.45.254:45131
I20260806 22:44:56.260998 23650 heartbeater.cc:464] Registering TS with master...
I20260806 22:44:56.261196 23650 heartbeater.cc:511] Master 127.18.45.254:45131 requested a full tablet report, sending...
I20260806 22:44:56.261652 22964 ts_manager.cc:194] Registered new tserver with Master: 8b4e88ab177c438ab6cf76d4fe99b8eb (127.18.45.195:34295)
I20260806 22:44:56.262094 22964 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.45.195@KRBTEST.COM'} at 127.18.45.195:56559
I20260806 22:44:56.264506 18615 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 22:44:56 dist-test-slave-h3ft krb5kdc[22894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056296, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 06 22:44:56 dist-test-slave-h3ft krb5kdc[22894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056296, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:44:56.293861 23023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:56.285280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57242 (local address 127.18.45.254:45131)
0806 22:44:56.285382 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:56.285385 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:56.285467 (+ 82us) server_negotiation.cc:407] Connection header received
0806 22:44:56.285529 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:56.285529 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:56.285542 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:56.285624 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:56.286299 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:56.286848 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:56.287680 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:56.287882 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:56.287940 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:56.290541 (+ 2601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:56.290543 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:56.290543 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:56.290558 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:56.292221 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:56.292707 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:56.292708 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:56.292709 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:56.292779 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:56.293113 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:56.293114 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:56.293115 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:56.293317 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:56.293415 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:56.293618 (+ 203us) server_negotiation.cc:299] Negotiation successful
0806 22:44:56.293703 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260806 22:44:56.294202 22964 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:57242
I20260806 22:44:56.294868 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 23057
I20260806 22:44:56.308439 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 23265
I20260806 22:44:56.326293 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 23468
I20260806 22:44:56.345724 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 22916
2026-08-06T22:44:56Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2360 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:44:56 dist-test-slave-h3ft krb5kdc[23670](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:44:56 dist-test-slave-h3ft krb5kdc[23670](info): set up 2 sockets
Aug 06 22:44:56 dist-test-slave-h3ft krb5kdc[23670](info): commencing operation
krb5kdc: starting...
W20260806 22:44:57.622485 18615 mini_kdc.cc:121] Time spent starting KDC: real 1.248s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Aug 06 22:44:57 dist-test-slave-h3ft krb5kdc[23670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056297, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:44:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:44:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.45.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab.
Entry for principal kudu/127.18.45.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab.
I20260806 22:44:57.774586 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:39471
--webserver_interface=127.18.45.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:35717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.45.254:39471
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:44:57.875218 23695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:57.875502 23695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:57.875530 23695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:57.876770 23695 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:44:57.876811 23695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:57.876827 23695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:57.876840 23695 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:44:57.876855 23695 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:44:57.878715 23695 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:35717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.45.254:39471
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:39471
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.45.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23695
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:44:57.878986 23695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:57.879297 23695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:57.885253 23705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:57.885397 23695 server_base.cc:1034] running on GCE node
W20260806 22:44:57.885278 23702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:57.885260 23700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:57.885856 23695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:57.886238 23695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:57.887421 23695 hybrid_clock.cc:648] HybridClock initialized: now 1786056297887406 us; error 47 us; skew 500 ppm
Aug 06 22:44:57 dist-test-slave-h3ft krb5kdc[23670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056297, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:57.890311 23695 init.cc:377] Logged in from keytab as kudu/127.18.45.254@KRBTEST.COM (short username kudu)
I20260806 22:44:57.890822 23695 webserver.cc:460] Webserver started at http://127.18.45.254:44207/ using document root <none> and password file <none>
I20260806 22:44:57.891162 23695 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:57.891238 23695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:57.891417 23695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:44:57.892738 23695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/minicluster-data/master-0/data/instance:
uuid: "43387e54ae0c46228817f1b7a5b1ffd4"
format_stamp: "Formatted at 2026-08-06 22:44:57 on dist-test-slave-h3ft"
server_key: "caee799854de56fec2ab7409144d5880"
server_key_iv: "c3293b81a43cf959ce6c21da5ae0bbb1"
server_key_version: "encryptionkey@0"
I20260806 22:44:57.893177 23695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/minicluster-data/master-0/wal/instance:
uuid: "43387e54ae0c46228817f1b7a5b1ffd4"
format_stamp: "Formatted at 2026-08-06 22:44:57 on dist-test-slave-h3ft"
server_key: "caee799854de56fec2ab7409144d5880"
server_key_iv: "c3293b81a43cf959ce6c21da5ae0bbb1"
server_key_version: "encryptionkey@0"
I20260806 22:44:57.894774 23695 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:44:57.895710 23715 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:57.895920 23695 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:44:57.896067 23695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/minicluster-data/master-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/minicluster-data/master-0/wal
uuid: "43387e54ae0c46228817f1b7a5b1ffd4"
format_stamp: "Formatted at 2026-08-06 22:44:57 on dist-test-slave-h3ft"
server_key: "caee799854de56fec2ab7409144d5880"
server_key_iv: "c3293b81a43cf959ce6c21da5ae0bbb1"
server_key_version: "encryptionkey@0"
I20260806 22:44:57.896155 23695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:57.918607 23695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:57.920982 23695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:57.921183 23695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:57.926117 23695 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.254:39471
I20260806 22:44:57.926226 23801 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.254:39471 every 8 connection(s)
I20260806 22:44:57.926815 23695 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
I20260806 22:44:57.927769 23803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:44:57.930807 23803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43387e54ae0c46228817f1b7a5b1ffd4: Bootstrap starting.
I20260806 22:44:57.931645 23803 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 43387e54ae0c46228817f1b7a5b1ffd4: Neither blocks nor log segments found. Creating new log.
I20260806 22:44:57.932068 23803 log.cc:824] T 00000000000000000000000000000000 P 43387e54ae0c46228817f1b7a5b1ffd4: Log is configured to *not* fsync() on all Append() calls
I20260806 22:44:57.932459 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 23695
I20260806 22:44:57.932595 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/minicluster-data/master-0/wal/instance
I20260806 22:44:57.932891 18615 external_mini_cluster.cc:1428] Setting key e0c453b27ef47cd4e8815e233e6772aa
I20260806 22:44:57.933209 23803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43387e54ae0c46228817f1b7a5b1ffd4: No bootstrap required, opened a new log
I20260806 22:44:57.936311 23803 raft_consensus.cc:348] T 00000000000000000000000000000000 P 43387e54ae0c46228817f1b7a5b1ffd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43387e54ae0c46228817f1b7a5b1ffd4" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39471 } }
I20260806 22:44:57.936491 23803 raft_consensus.cc:374] T 00000000000000000000000000000000 P 43387e54ae0c46228817f1b7a5b1ffd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:44:57.936520 23803 raft_consensus.cc:729] T 00000000000000000000000000000000 P 43387e54ae0c46228817f1b7a5b1ffd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43387e54ae0c46228817f1b7a5b1ffd4, State: Initialized, Role: FOLLOWER
I20260806 22:44:57.936709 23803 consensus_queue.cc:260] T 00000000000000000000000000000000 P 43387e54ae0c46228817f1b7a5b1ffd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43387e54ae0c46228817f1b7a5b1ffd4" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39471 } }
I20260806 22:44:57.936803 23803 raft_consensus.cc:388] T 00000000000000000000000000000000 P 43387e54ae0c46228817f1b7a5b1ffd4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:44:57.936857 23803 raft_consensus.cc:482] T 00000000000000000000000000000000 P 43387e54ae0c46228817f1b7a5b1ffd4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:44:57.936914 23803 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 43387e54ae0c46228817f1b7a5b1ffd4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:44:57.937630 23803 raft_consensus.cc:504] T 00000000000000000000000000000000 P 43387e54ae0c46228817f1b7a5b1ffd4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43387e54ae0c46228817f1b7a5b1ffd4" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39471 } }
I20260806 22:44:57.937784 23803 leader_election.cc:302] T 00000000000000000000000000000000 P 43387e54ae0c46228817f1b7a5b1ffd4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 43387e54ae0c46228817f1b7a5b1ffd4; no voters:
I20260806 22:44:57.938035 23803 leader_election.cc:288] T 00000000000000000000000000000000 P 43387e54ae0c46228817f1b7a5b1ffd4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:44:57.938071 23808 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 43387e54ae0c46228817f1b7a5b1ffd4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:44:57.938225 23808 raft_consensus.cc:686] T 00000000000000000000000000000000 P 43387e54ae0c46228817f1b7a5b1ffd4 [term 1 LEADER]: Becoming Leader. State: Replica: 43387e54ae0c46228817f1b7a5b1ffd4, State: Running, Role: LEADER
I20260806 22:44:57.938426 23803 sys_catalog.cc:564] T 00000000000000000000000000000000 P 43387e54ae0c46228817f1b7a5b1ffd4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:44:57.938395 23808 consensus_queue.cc:237] T 00000000000000000000000000000000 P 43387e54ae0c46228817f1b7a5b1ffd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43387e54ae0c46228817f1b7a5b1ffd4" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39471 } }
I20260806 22:44:57.939133 23813 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43387e54ae0c46228817f1b7a5b1ffd4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 43387e54ae0c46228817f1b7a5b1ffd4. Latest consensus state: current_term: 1 leader_uuid: "43387e54ae0c46228817f1b7a5b1ffd4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43387e54ae0c46228817f1b7a5b1ffd4" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39471 } } }
I20260806 22:44:57.939314 23813 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43387e54ae0c46228817f1b7a5b1ffd4 [sys.catalog]: This master's current role is: LEADER
I20260806 22:44:57.939635 23812 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43387e54ae0c46228817f1b7a5b1ffd4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "43387e54ae0c46228817f1b7a5b1ffd4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43387e54ae0c46228817f1b7a5b1ffd4" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39471 } } }
I20260806 22:44:57.939812 23812 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43387e54ae0c46228817f1b7a5b1ffd4 [sys.catalog]: This master's current role is: LEADER
I20260806 22:44:57.939723 23817 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:44:57.940577 23817 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 22:44:57 dist-test-slave-h3ft krb5kdc[23670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056297, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:44:57.942903 23817 catalog_manager.cc:1347] Generated new cluster ID: ad044f473ae345aea28bc4545fc15cad
I20260806 22:44:57.942960 23817 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:44:57.947214 23807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:57.933640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33702 (local address 127.18.45.254:39471)
0806 22:44:57.933861 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:57.933873 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:57.933935 (+ 62us) server_negotiation.cc:407] Connection header received
0806 22:44:57.934813 (+ 878us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:57.934822 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:57.934909 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:57.935176 (+ 267us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:57.935953 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:57.937168 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:57.938209 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:57.938796 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:57.938943 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:57.943171 (+ 4228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:57.943175 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:57.943176 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:57.943189 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:57.945314 (+ 2125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:57.945774 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:57.945778 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:57.945779 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:57.945898 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:57.946153 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:57.946154 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:57.946154 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:57.946457 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:57.946661 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:57.946747 (+ 86us) server_negotiation.cc:299] Negotiation successful
0806 22:44:57.946923 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":100,"threads_started":1}
I20260806 22:44:57.954098 23817 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:44:57.954898 23817 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:44:57.966451 23817 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 43387e54ae0c46228817f1b7a5b1ffd4: Generated new TSK 0
I20260806 22:44:57.966763 23817 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.45.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab.
Entry for principal kudu/127.18.45.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab.
I20260806 22:44:58.011689 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.193:0
--local_ip_for_outbound_sockets=127.18.45.193
--webserver_interface=127.18.45.193
--webserver_port=0
--tserver_master_addrs=127.18.45.254:39471
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab
--principal=kudu/127.18.45.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:35717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:44:58.124418 23840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:58.124715 23840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:58.124745 23840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:58.126016 23840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:58.126057 23840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:58.126085 23840 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.193
I20260806 22:44:58.127852 23840 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:35717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab
--principal=kudu/127.18.45.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.45.193
--webserver_port=0
--tserver_master_addrs=127.18.45.254:39471
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23840
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.193
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:44:58.128104 23840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:58.128348 23840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:58.134490 23857 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:58.134517 23848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:58.134550 23850 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:58.134719 23840 server_base.cc:1034] running on GCE node
I20260806 22:44:58.135077 23840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:58.135373 23840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:58.136557 23840 hybrid_clock.cc:648] HybridClock initialized: now 1786056298136533 us; error 39 us; skew 500 ppm
Aug 06 22:44:58 dist-test-slave-h3ft krb5kdc[23670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056298, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:58.139811 23840 init.cc:377] Logged in from keytab as kudu/127.18.45.193@KRBTEST.COM (short username kudu)
I20260806 22:44:58.140365 23840 webserver.cc:460] Webserver started at http://127.18.45.193:36203/ using document root <none> and password file <none>
I20260806 22:44:58.140631 23840 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:58.140704 23840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:58.140838 23840 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:44:58.142045 23840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/minicluster-data/ts-0/data/instance:
uuid: "4576e4bdff5c4b6f82b3bf9693d29995"
format_stamp: "Formatted at 2026-08-06 22:44:58 on dist-test-slave-h3ft"
server_key: "7def2682201a88ba88bbfc018f106652"
server_key_iv: "a634662ed3e9853b86f057218ff7496c"
server_key_version: "encryptionkey@0"
I20260806 22:44:58.142412 23840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/minicluster-data/ts-0/wal/instance:
uuid: "4576e4bdff5c4b6f82b3bf9693d29995"
format_stamp: "Formatted at 2026-08-06 22:44:58 on dist-test-slave-h3ft"
server_key: "7def2682201a88ba88bbfc018f106652"
server_key_iv: "a634662ed3e9853b86f057218ff7496c"
server_key_version: "encryptionkey@0"
I20260806 22:44:58.144027 23840 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:44:58.144912 23866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:58.145128 23840 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:44:58.145242 23840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/minicluster-data/ts-0/wal
uuid: "4576e4bdff5c4b6f82b3bf9693d29995"
format_stamp: "Formatted at 2026-08-06 22:44:58 on dist-test-slave-h3ft"
server_key: "7def2682201a88ba88bbfc018f106652"
server_key_iv: "a634662ed3e9853b86f057218ff7496c"
server_key_version: "encryptionkey@0"
I20260806 22:44:58.145326 23840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:58.157454 23840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:58.159772 23840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:58.159931 23840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:58.160240 23840 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:44:58.160660 23840 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:44:58.160720 23840 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:58.160771 23840 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:44:58.160801 23840 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:58.168351 23840 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.193:36995
I20260806 22:44:58.168411 24028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.193:36995 every 8 connection(s)
I20260806 22:44:58.168793 23840 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
I20260806 22:44:58.170214 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 23840
I20260806 22:44:58.170351 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/minicluster-data/ts-0/wal/instance
I20260806 22:44:58.170585 18615 external_mini_cluster.cc:1428] Setting key 57c50ca80a30a290a291d62ba53a4c78
Aug 06 22:44:58 dist-test-slave-h3ft krb5kdc[23670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056298, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.193@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:44:58.181175 23807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:58.169684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:47673 (local address 127.18.45.254:39471)
0806 22:44:58.169766 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:58.169768 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:58.170030 (+ 262us) server_negotiation.cc:407] Connection header received
0806 22:44:58.171431 (+ 1401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:58.171433 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:58.171448 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:58.171511 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:58.172273 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:58.172859 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:58.174041 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:58.174391 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:58.174439 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:58.177429 (+ 2990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:58.177430 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:58.177431 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:58.177440 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:58.179420 (+ 1980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:58.179895 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:58.179897 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:58.179898 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:58.179981 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:58.180304 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:58.180305 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:58.180305 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:58.180495 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:58.180586 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:58.180908 (+ 322us) server_negotiation.cc:299] Negotiation successful
0806 22:44:58.181033 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 22:44:58.181347 24032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:58.169643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39471 (local address 127.18.45.193:47673)
0806 22:44:58.169879 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:58.169923 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:58.171313 (+ 1390us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:58.171610 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:58.171614 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:58.171925 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:58.172170 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:58.172174 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:58.172972 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:58.172972 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:58.173917 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:58.173921 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:58.174564 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:58.174565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:58.174735 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:58.175427 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:44:58.175440 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:58.177315 (+ 1875us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:58.179528 (+ 2213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:58.179535 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:58.179537 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:58.179813 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:58.180093 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:58.180094 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:58.180094 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:58.180221 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:58.180584 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:58.180591 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:58.180751 (+ 160us) client_negotiation.cc:769] Sending connection context
0806 22:44:58.180852 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 22:44:58.181014 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":118,"threads_started":1}
I20260806 22:44:58.182016 24029 heartbeater.cc:347] Connected to a master server at 127.18.45.254:39471
I20260806 22:44:58.182132 24029 heartbeater.cc:464] Registering TS with master...
I20260806 22:44:58.182374 24029 heartbeater.cc:511] Master 127.18.45.254:39471 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.18.45.194@KRBTEST.COM; defaulting to no policy
I20260806 22:44:58.183207 23743 ts_manager.cc:194] Registered new tserver with Master: 4576e4bdff5c4b6f82b3bf9693d29995 (127.18.45.193:36995)
I20260806 22:44:58.183919 23743 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.45.193@KRBTEST.COM'} at 127.18.45.193:47673
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab.
Entry for principal kudu/127.18.45.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab.
I20260806 22:44:58.205662 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.194:0
--local_ip_for_outbound_sockets=127.18.45.194
--webserver_interface=127.18.45.194
--webserver_port=0
--tserver_master_addrs=127.18.45.254:39471
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab
--principal=kudu/127.18.45.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:35717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:44:58.308771 24035 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:58.309041 24035 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:58.309079 24035 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:58.310289 24035 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:58.310328 24035 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:58.310354 24035 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.194
I20260806 22:44:58.312008 24035 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:35717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab
--principal=kudu/127.18.45.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.45.194
--webserver_port=0
--tserver_master_addrs=127.18.45.254:39471
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24035
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.194
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:44:58.312317 24035 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:58.312623 24035 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:58.318692 24047 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:58.318697 24046 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:58.318697 24050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:58.319118 24035 server_base.cc:1034] running on GCE node
I20260806 22:44:58.319394 24035 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:58.319679 24035 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:58.320868 24035 hybrid_clock.cc:648] HybridClock initialized: now 1786056298320862 us; error 42 us; skew 500 ppm
Aug 06 22:44:58 dist-test-slave-h3ft krb5kdc[23670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056298, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:58.324314 24035 init.cc:377] Logged in from keytab as kudu/127.18.45.194@KRBTEST.COM (short username kudu)
I20260806 22:44:58.324863 24035 webserver.cc:460] Webserver started at http://127.18.45.194:43127/ using document root <none> and password file <none>
I20260806 22:44:58.325163 24035 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:58.325212 24035 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:58.325374 24035 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:44:58.326632 24035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/minicluster-data/ts-1/data/instance:
uuid: "2621a2c1bc54464785ea84873a250ca6"
format_stamp: "Formatted at 2026-08-06 22:44:58 on dist-test-slave-h3ft"
server_key: "5ccae70b3c94e186dd04c51793b9fdfb"
server_key_iv: "2a5044a91d1b2163b728082aaab220e7"
server_key_version: "encryptionkey@0"
I20260806 22:44:58.327109 24035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/minicluster-data/ts-1/wal/instance:
uuid: "2621a2c1bc54464785ea84873a250ca6"
format_stamp: "Formatted at 2026-08-06 22:44:58 on dist-test-slave-h3ft"
server_key: "5ccae70b3c94e186dd04c51793b9fdfb"
server_key_iv: "2a5044a91d1b2163b728082aaab220e7"
server_key_version: "encryptionkey@0"
I20260806 22:44:58.328671 24035 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:44:58.329543 24062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:58.329743 24035 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:44:58.329847 24035 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/minicluster-data/ts-1/wal
uuid: "2621a2c1bc54464785ea84873a250ca6"
format_stamp: "Formatted at 2026-08-06 22:44:58 on dist-test-slave-h3ft"
server_key: "5ccae70b3c94e186dd04c51793b9fdfb"
server_key_iv: "2a5044a91d1b2163b728082aaab220e7"
server_key_version: "encryptionkey@0"
I20260806 22:44:58.329941 24035 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:58.346662 24035 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:58.349205 24035 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:58.349393 24035 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:58.349766 24035 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:44:58.350162 24035 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:44:58.350221 24035 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:58.350270 24035 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:44:58.350298 24035 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:58.357831 24035 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.194:39847
I20260806 22:44:58.357925 24219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.194:39847 every 8 connection(s)
I20260806 22:44:58.358317 24035 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
I20260806 22:44:58.363205 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 24035
I20260806 22:44:58.363343 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/minicluster-data/ts-1/wal/instance
I20260806 22:44:58.363570 18615 external_mini_cluster.cc:1428] Setting key 76e0cd2116becbacf72eef3db993d7d1
Aug 06 22:44:58 dist-test-slave-h3ft krb5kdc[23670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056298, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.194@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:44:58.370708 23807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:58.359349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:34539 (local address 127.18.45.254:39471)
0806 22:44:58.359452 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:58.359455 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:58.359724 (+ 269us) server_negotiation.cc:407] Connection header received
0806 22:44:58.360910 (+ 1186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:58.360911 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:58.360930 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:58.360988 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:58.361773 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:58.362333 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:58.363859 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:58.364188 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:58.364261 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:58.367285 (+ 3024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:58.367287 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:58.367287 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:58.367298 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:58.369098 (+ 1800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:58.369555 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:58.369556 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:58.369556 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:58.369631 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:58.369884 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:58.369885 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:58.369885 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:58.370073 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:58.370147 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:58.370427 (+ 280us) server_negotiation.cc:299] Negotiation successful
0806 22:44:58.370545 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 22:44:58.370756 24223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:58.359320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39471 (local address 127.18.45.194:34539)
0806 22:44:58.359575 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:58.359621 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:58.360770 (+ 1149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:58.361081 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:58.361086 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:58.361415 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:58.361663 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:58.361669 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:58.362458 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:58.362459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:58.363729 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:58.363733 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:58.364278 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:58.364279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:58.364466 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:58.365211 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:44:58.365222 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:58.367175 (+ 1953us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:58.369195 (+ 2020us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:58.369204 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:58.369206 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:58.369470 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:58.369702 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:58.369703 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:58.369703 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:58.369818 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:58.370138 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:58.370143 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:58.370287 (+ 144us) client_negotiation.cc:769] Sending connection context
0806 22:44:58.370379 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 22:44:58.370545 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":131,"threads_started":1}
I20260806 22:44:58.371402 24220 heartbeater.cc:347] Connected to a master server at 127.18.45.254:39471
I20260806 22:44:58.371546 24220 heartbeater.cc:464] Registering TS with master...
I20260806 22:44:58.371791 24220 heartbeater.cc:511] Master 127.18.45.254:39471 requested a full tablet report, sending...
I20260806 22:44:58.372352 23743 ts_manager.cc:194] Registered new tserver with Master: 2621a2c1bc54464785ea84873a250ca6 (127.18.45.194:39847)
I20260806 22:44:58.372984 23743 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.45.194@KRBTEST.COM'} at 127.18.45.194:34539
WARNING: no policy specified for kudu/127.18.45.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab.
Entry for principal kudu/127.18.45.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab.
I20260806 22:44:58.399266 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.195:0
--local_ip_for_outbound_sockets=127.18.45.195
--webserver_interface=127.18.45.195
--webserver_port=0
--tserver_master_addrs=127.18.45.254:39471
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab
--principal=kudu/127.18.45.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:35717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:44:58.498862 24227 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:44:58.499217 24227 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:44:58.499245 24227 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:44:58.500522 24227 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:44:58.500569 24227 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:44:58.500597 24227 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.195
I20260806 22:44:58.502274 24227 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:35717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab
--principal=kudu/127.18.45.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.45.195
--webserver_port=0
--tserver_master_addrs=127.18.45.254:39471
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24227
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.195
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:44:58.502580 24227 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:44:58.502857 24227 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:44:58.508900 24239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:58.508872 24234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:44:58.508891 24237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:44:58.509245 24227 server_base.cc:1034] running on GCE node
I20260806 22:44:58.509507 24227 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:44:58.509783 24227 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:44:58.510936 24227 hybrid_clock.cc:648] HybridClock initialized: now 1786056298510914 us; error 42 us; skew 500 ppm
Aug 06 22:44:58 dist-test-slave-h3ft krb5kdc[23670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056298, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:44:58.514143 24227 init.cc:377] Logged in from keytab as kudu/127.18.45.195@KRBTEST.COM (short username kudu)
I20260806 22:44:58.514737 24227 webserver.cc:460] Webserver started at http://127.18.45.195:38865/ using document root <none> and password file <none>
I20260806 22:44:58.515115 24227 fs_manager.cc:359] Metadata directory not provided
I20260806 22:44:58.515198 24227 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:44:58.515380 24227 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:44:58.516633 24227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/minicluster-data/ts-2/data/instance:
uuid: "985c8c84f44d44689ccd28ba2d7509e3"
format_stamp: "Formatted at 2026-08-06 22:44:58 on dist-test-slave-h3ft"
server_key: "434e1ef2d765fab9f9c4ccab047d2b5d"
server_key_iv: "ec7533d2ffb4a052cbf33efd0c56a5f0"
server_key_version: "encryptionkey@0"
I20260806 22:44:58.517071 24227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/minicluster-data/ts-2/wal/instance:
uuid: "985c8c84f44d44689ccd28ba2d7509e3"
format_stamp: "Formatted at 2026-08-06 22:44:58 on dist-test-slave-h3ft"
server_key: "434e1ef2d765fab9f9c4ccab047d2b5d"
server_key_iv: "ec7533d2ffb4a052cbf33efd0c56a5f0"
server_key_version: "encryptionkey@0"
I20260806 22:44:58.518628 24227 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:44:58.519608 24248 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:58.519829 24227 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:44:58.519949 24227 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/minicluster-data/ts-2/wal
uuid: "985c8c84f44d44689ccd28ba2d7509e3"
format_stamp: "Formatted at 2026-08-06 22:44:58 on dist-test-slave-h3ft"
server_key: "434e1ef2d765fab9f9c4ccab047d2b5d"
server_key_iv: "ec7533d2ffb4a052cbf33efd0c56a5f0"
server_key_version: "encryptionkey@0"
I20260806 22:44:58.520037 24227 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:44:58.548790 24227 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:44:58.551753 24227 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:44:58.551986 24227 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:44:58.552356 24227 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:44:58.552804 24227 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:44:58.552878 24227 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:58.552925 24227 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:44:58.552978 24227 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:44:58.561352 24227 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.195:42941
I20260806 22:44:58.561417 24419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.195:42941 every 8 connection(s)
I20260806 22:44:58.561849 24227 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
I20260806 22:44:58.567281 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 24227
I20260806 22:44:58.567412 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786056275803658-18615-0/minicluster-data/ts-2/wal/instance
I20260806 22:44:58.567656 18615 external_mini_cluster.cc:1428] Setting key 696434d8fd4fd093d3eee6812e570177
Aug 06 22:44:58 dist-test-slave-h3ft krb5kdc[23670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056298, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.195@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:44:58.573855 23807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:58.562804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.195:41631 (local address 127.18.45.254:39471)
0806 22:44:58.562884 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:58.562888 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:58.563158 (+ 270us) server_negotiation.cc:407] Connection header received
0806 22:44:58.564458 (+ 1300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:58.564459 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:58.564472 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:58.564538 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:58.565366 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:58.565997 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:58.567293 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:58.567627 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:58.567720 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:58.570716 (+ 2996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:58.570717 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:58.570718 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:58.570726 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:58.572365 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:58.572832 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:58.572833 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:58.572834 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:58.572897 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:58.573200 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:58.573200 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:58.573201 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:58.573328 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:58.573383 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:58.573688 (+ 305us) server_negotiation.cc:299] Negotiation successful
0806 22:44:58.573771 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 22:44:58.574143 24422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:58.562744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39471 (local address 127.18.45.195:41631)
0806 22:44:58.562951 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0806 22:44:58.563031 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0806 22:44:58.564343 (+ 1312us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:44:58.564611 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:44:58.564616 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:44:58.564990 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:44:58.565246 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:58.565252 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:58.566109 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:58.566110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:58.567187 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:44:58.567190 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:44:58.567787 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:44:58.567789 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:44:58.567992 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:58.568683 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:44:58.568694 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:44:58.570604 (+ 1910us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:44:58.572459 (+ 1855us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:58.572468 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:58.572471 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:58.572760 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:58.572968 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:44:58.572969 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:44:58.572969 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:44:58.573141 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:44:58.573387 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:44:58.573394 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:44:58.573554 (+ 160us) client_negotiation.cc:769] Sending connection context
0806 22:44:58.573654 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 22:44:58.573812 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":117,"threads_started":1}
I20260806 22:44:58.574937 24420 heartbeater.cc:347] Connected to a master server at 127.18.45.254:39471
I20260806 22:44:58.575142 24420 heartbeater.cc:464] Registering TS with master...
I20260806 22:44:58.575415 24420 heartbeater.cc:511] Master 127.18.45.254:39471 requested a full tablet report, sending...
I20260806 22:44:58.575994 23743 ts_manager.cc:194] Registered new tserver with Master: 985c8c84f44d44689ccd28ba2d7509e3 (127.18.45.195:42941)
I20260806 22:44:58.576413 23743 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.45.195@KRBTEST.COM'} at 127.18.45.195:41631
I20260806 22:44:58.580636 18615 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 22:44:58 dist-test-slave-h3ft krb5kdc[23670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056298, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 22:44:58 dist-test-slave-h3ft krb5kdc[23670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056298, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.45.193@KRBTEST.COM
I20260806 22:44:58.610085 24428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:58.601551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39528 (local address 127.18.45.193:36995)
0806 22:44:58.601777 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:58.601784 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:58.601797 (+ 13us) server_negotiation.cc:407] Connection header received
0806 22:44:58.601833 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:58.601835 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:58.601854 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:58.601933 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:58.602571 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:58.603465 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:58.604244 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:58.604603 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:58.604664 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:58.607260 (+ 2596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:58.607271 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:58.607272 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:58.607280 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:58.608829 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:58.609317 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:58.609321 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:58.609321 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:58.609386 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:58.609560 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:58.609561 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:58.609561 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:58.609722 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:58.609857 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:58.609897 (+ 40us) server_negotiation.cc:299] Negotiation successful
0806 22:44:58.609965 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":124,"threads_started":1}
Aug 06 22:44:58 dist-test-slave-h3ft krb5kdc[23670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056298, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:44:58.619107 23807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:44:58.611887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48248 (local address 127.18.45.254:39471)
0806 22:44:58.611963 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 22:44:58.611965 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:44:58.611977 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:44:58.612044 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:44:58.612045 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:44:58.612059 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:44:58.612114 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:44:58.612582 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:58.613103 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:58.613780 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:44:58.613994 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:44:58.614063 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:44:58.616492 (+ 2429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:44:58.616494 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:44:58.616495 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:44:58.616503 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:44:58.617783 (+ 1280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:58.618144 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:58.618146 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:58.618147 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:58.618215 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:44:58.618429 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:44:58.618430 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:44:58.618430 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:44:58.618628 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:44:58.618751 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 22:44:58.618852 (+ 101us) server_negotiation.cc:299] Negotiation successful
0806 22:44:58.618944 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
W20260806 22:44:58.619415 23743 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48248
I20260806 22:44:58.619951 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 23840
I20260806 22:44:58.632470 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 24035
I20260806 22:44:58.644953 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 24227
I20260806 22:44:58.659602 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 23695
2026-08-06T22:44:58Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2315 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:44:58 dist-test-slave-h3ft krb5kdc[24435](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:44:58 dist-test-slave-h3ft krb5kdc[24435](info): set up 2 sockets
Aug 06 22:44:58 dist-test-slave-h3ft krb5kdc[24435](info): commencing operation
krb5kdc: starting...
W20260806 22:44:59.919486 18615 mini_kdc.cc:121] Time spent starting KDC: real 1.229s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Aug 06 22:45:00 dist-test-slave-h3ft krb5kdc[24435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056300, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:45:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:45:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.45.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab.
Entry for principal kudu/127.18.45.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab.
I20260806 22:45:00.079324 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:45665
--webserver_interface=127.18.45.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:38945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.45.254:45665
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:45:00.180533 24452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:45:00.180837 24452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:45:00.180867 24452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:45:00.182199 24452 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:45:00.182246 24452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:45:00.182261 24452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:45:00.182274 24452 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:45:00.182286 24452 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:45:00.183998 24452 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:38945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.45.254:45665
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:45665
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.45.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24452
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:45:00.184274 24452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:45:00.184520 24452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:45:00.191385 24460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:00.191563 24452 server_base.cc:1034] running on GCE node
W20260806 22:45:00.191680 24458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:00.191385 24457 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:00.192101 24452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:45:00.192526 24452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:45:00.193722 24452 hybrid_clock.cc:648] HybridClock initialized: now 1786056300193679 us; error 65 us; skew 500 ppm
Aug 06 22:45:00 dist-test-slave-h3ft krb5kdc[24435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056300, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:45:00.202637 24452 init.cc:377] Logged in from keytab as kudu/127.18.45.254@KRBTEST.COM (short username kudu)
I20260806 22:45:00.203249 24452 fs_manager.cc:359] Metadata directory not provided
I20260806 22:45:00.203339 24452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:45:00.203527 24452 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:45:00.204893 24452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/minicluster-data/master-0/data/instance:
uuid: "2289b5a3377b434e9fda5c21f411a194"
format_stamp: "Formatted at 2026-08-06 22:45:00 on dist-test-slave-h3ft"
server_key: "31a46cae584ef30691cd8d5c5fc01d2b"
server_key_iv: "bb23a31b1861b7a19af10f8b774760e1"
server_key_version: "encryptionkey@0"
I20260806 22:45:00.205361 24452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/minicluster-data/master-0/wal/instance:
uuid: "2289b5a3377b434e9fda5c21f411a194"
format_stamp: "Formatted at 2026-08-06 22:45:00 on dist-test-slave-h3ft"
server_key: "31a46cae584ef30691cd8d5c5fc01d2b"
server_key_iv: "bb23a31b1861b7a19af10f8b774760e1"
server_key_version: "encryptionkey@0"
I20260806 22:45:00.207073 24452 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260806 22:45:00.208034 24471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:00.208280 24452 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:45:00.208410 24452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/minicluster-data/master-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/minicluster-data/master-0/wal
uuid: "2289b5a3377b434e9fda5c21f411a194"
format_stamp: "Formatted at 2026-08-06 22:45:00 on dist-test-slave-h3ft"
server_key: "31a46cae584ef30691cd8d5c5fc01d2b"
server_key_iv: "bb23a31b1861b7a19af10f8b774760e1"
server_key_version: "encryptionkey@0"
I20260806 22:45:00.208499 24452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:45:00.225525 24452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:45:00.228188 24452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:45:00.228405 24452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:45:00.233736 24452 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.254:45665
I20260806 22:45:00.233767 24545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.254:45665 every 8 connection(s)
I20260806 22:45:00.234148 24452 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
I20260806 22:45:00.235482 24546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:45:00.237943 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 24452
I20260806 22:45:00.238101 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/minicluster-data/master-0/wal/instance
I20260806 22:45:00.238328 18615 external_mini_cluster.cc:1428] Setting key 1b8e46847264d92cbbe7a77675ea3701
I20260806 22:45:00.238790 24546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2289b5a3377b434e9fda5c21f411a194: Bootstrap starting.
I20260806 22:45:00.239915 24546 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2289b5a3377b434e9fda5c21f411a194: Neither blocks nor log segments found. Creating new log.
I20260806 22:45:00.240299 24546 log.cc:824] T 00000000000000000000000000000000 P 2289b5a3377b434e9fda5c21f411a194: Log is configured to *not* fsync() on all Append() calls
I20260806 22:45:00.241146 24546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2289b5a3377b434e9fda5c21f411a194: No bootstrap required, opened a new log
I20260806 22:45:00.243290 24546 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2289b5a3377b434e9fda5c21f411a194 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2289b5a3377b434e9fda5c21f411a194" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 45665 } }
I20260806 22:45:00.243489 24546 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2289b5a3377b434e9fda5c21f411a194 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:45:00.243570 24546 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2289b5a3377b434e9fda5c21f411a194 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2289b5a3377b434e9fda5c21f411a194, State: Initialized, Role: FOLLOWER
I20260806 22:45:00.243747 24546 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2289b5a3377b434e9fda5c21f411a194 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2289b5a3377b434e9fda5c21f411a194" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 45665 } }
I20260806 22:45:00.243860 24546 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2289b5a3377b434e9fda5c21f411a194 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:45:00.243901 24546 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2289b5a3377b434e9fda5c21f411a194 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:45:00.243959 24546 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2289b5a3377b434e9fda5c21f411a194 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:45:00.244737 24546 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2289b5a3377b434e9fda5c21f411a194 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2289b5a3377b434e9fda5c21f411a194" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 45665 } }
I20260806 22:45:00.244846 24546 leader_election.cc:302] T 00000000000000000000000000000000 P 2289b5a3377b434e9fda5c21f411a194 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2289b5a3377b434e9fda5c21f411a194; no voters:
I20260806 22:45:00.245118 24546 leader_election.cc:288] T 00000000000000000000000000000000 P 2289b5a3377b434e9fda5c21f411a194 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:45:00.245189 24553 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2289b5a3377b434e9fda5c21f411a194 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:45:00.245396 24553 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2289b5a3377b434e9fda5c21f411a194 [term 1 LEADER]: Becoming Leader. State: Replica: 2289b5a3377b434e9fda5c21f411a194, State: Running, Role: LEADER
I20260806 22:45:00.245476 24546 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2289b5a3377b434e9fda5c21f411a194 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:45:00.245572 24553 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2289b5a3377b434e9fda5c21f411a194 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2289b5a3377b434e9fda5c21f411a194" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 45665 } }
I20260806 22:45:00.246014 24553 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2289b5a3377b434e9fda5c21f411a194 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2289b5a3377b434e9fda5c21f411a194. Latest consensus state: current_term: 1 leader_uuid: "2289b5a3377b434e9fda5c21f411a194" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2289b5a3377b434e9fda5c21f411a194" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 45665 } } }
I20260806 22:45:00.246098 24553 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2289b5a3377b434e9fda5c21f411a194 [sys.catalog]: This master's current role is: LEADER
Aug 06 22:45:00 dist-test-slave-h3ft krb5kdc[24435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056300, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:45:00.246366 24554 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2289b5a3377b434e9fda5c21f411a194 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2289b5a3377b434e9fda5c21f411a194" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2289b5a3377b434e9fda5c21f411a194" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 45665 } } }
I20260806 22:45:00.246474 24554 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2289b5a3377b434e9fda5c21f411a194 [sys.catalog]: This master's current role is: LEADER
I20260806 22:45:00.246847 24565 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:45:00.247709 24565 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:45:00.249809 24565 catalog_manager.cc:1347] Generated new cluster ID: 432c00f791fc4a309bf14b31a73c9579
I20260806 22:45:00.249866 24565 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:45:00.251518 24549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:00.239024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38214 (local address 127.18.45.254:45665)
0806 22:45:00.239249 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:00.239260 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:00.239296 (+ 36us) server_negotiation.cc:407] Connection header received
0806 22:45:00.239879 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:00.239887 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:00.239945 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:00.240177 (+ 232us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:00.240968 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:00.242004 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:00.242844 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:00.243386 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:00.243520 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:00.246803 (+ 3283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:00.246808 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:00.246808 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:00.246825 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:00.249498 (+ 2673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:00.250018 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:00.250021 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:00.250022 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:00.250148 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:00.250414 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:00.250414 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:00.250415 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:00.250745 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:00.250896 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:00.251110 (+ 214us) server_negotiation.cc:299] Negotiation successful
0806 22:45:00.251264 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":108,"threads_started":1}
I20260806 22:45:00.257154 24565 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:45:00.257777 24565 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:45:00.271360 24565 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2289b5a3377b434e9fda5c21f411a194: Generated new TSK 0
I20260806 22:45:00.271639 24565 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.45.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab.
Entry for principal kudu/127.18.45.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab.
I20260806 22:45:00.311556 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.193:0
--local_ip_for_outbound_sockets=127.18.45.193
--webserver_interface=127.18.45.193
--webserver_port=0
--tserver_master_addrs=127.18.45.254:45665
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab
--principal=kudu/127.18.45.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:38945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:45:00.414062 24580 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:45:00.414340 24580 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:45:00.414372 24580 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:45:00.416085 24580 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:45:00.416158 24580 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:45:00.416193 24580 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.193
I20260806 22:45:00.418176 24580 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:38945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab
--principal=kudu/127.18.45.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.45.193
--webserver_port=0
--tserver_master_addrs=127.18.45.254:45665
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24580
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.193
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:45:00.418442 24580 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:45:00.418696 24580 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:45:00.425150 24590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:00.425103 24588 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:00.425143 24592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:00.425580 24580 server_base.cc:1034] running on GCE node
I20260806 22:45:00.425835 24580 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:45:00.426131 24580 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:45:00.427289 24580 hybrid_clock.cc:648] HybridClock initialized: now 1786056300427277 us; error 38 us; skew 500 ppm
Aug 06 22:45:00 dist-test-slave-h3ft krb5kdc[24435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056300, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:45:00.430735 24580 init.cc:377] Logged in from keytab as kudu/127.18.45.193@KRBTEST.COM (short username kudu)
I20260806 22:45:00.431330 24580 fs_manager.cc:359] Metadata directory not provided
I20260806 22:45:00.431422 24580 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:45:00.431618 24580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:45:00.432962 24580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/minicluster-data/ts-0/data/instance:
uuid: "e8d589e6c8a6423da76d4b3080cb651a"
format_stamp: "Formatted at 2026-08-06 22:45:00 on dist-test-slave-h3ft"
server_key: "7fd91e8c89b59c97006fe1bd316f7034"
server_key_iv: "b28e1baa6f0cfe9a615101caa7ed8719"
server_key_version: "encryptionkey@0"
I20260806 22:45:00.433404 24580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/minicluster-data/ts-0/wal/instance:
uuid: "e8d589e6c8a6423da76d4b3080cb651a"
format_stamp: "Formatted at 2026-08-06 22:45:00 on dist-test-slave-h3ft"
server_key: "7fd91e8c89b59c97006fe1bd316f7034"
server_key_iv: "b28e1baa6f0cfe9a615101caa7ed8719"
server_key_version: "encryptionkey@0"
I20260806 22:45:00.435097 24580 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260806 22:45:00.436070 24599 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:00.436287 24580 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:45:00.436412 24580 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/minicluster-data/ts-0/wal
uuid: "e8d589e6c8a6423da76d4b3080cb651a"
format_stamp: "Formatted at 2026-08-06 22:45:00 on dist-test-slave-h3ft"
server_key: "7fd91e8c89b59c97006fe1bd316f7034"
server_key_iv: "b28e1baa6f0cfe9a615101caa7ed8719"
server_key_version: "encryptionkey@0"
I20260806 22:45:00.436519 24580 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:45:00.449692 24580 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:45:00.452085 24580 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:45:00.452270 24580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:45:00.452610 24580 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:45:00.452996 24580 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:45:00.453053 24580 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:00.453100 24580 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:45:00.453145 24580 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:00.460819 24580 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.193:37491
I20260806 22:45:00.460863 24768 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.193:37491 every 8 connection(s)
I20260806 22:45:00.461200 24580 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
Aug 06 22:45:00 dist-test-slave-h3ft krb5kdc[24435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056300, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.193@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:45:00.470125 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 24580
I20260806 22:45:00.470299 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/minicluster-data/ts-0/wal/instance
I20260806 22:45:00.470602 18615 external_mini_cluster.cc:1428] Setting key 55f334a6a39fb6bd2a45cb971b455a1e
I20260806 22:45:00.472785 24549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:00.462131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:38475 (local address 127.18.45.254:45665)
0806 22:45:00.462215 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:00.462218 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:00.462449 (+ 231us) server_negotiation.cc:407] Connection header received
0806 22:45:00.463108 (+ 659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:00.463110 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:00.463125 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:00.463200 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:00.464314 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:00.464914 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:00.466003 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:00.466223 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:00.466281 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:00.468971 (+ 2690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:00.468974 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:00.468974 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:00.468983 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:00.470690 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:00.471354 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:00.471356 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:00.471356 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:00.471457 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:00.471868 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:00.471870 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:00.471870 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:00.472116 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:00.472200 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:00.472485 (+ 285us) server_negotiation.cc:299] Negotiation successful
0806 22:45:00.472629 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 22:45:00.472816 24771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:00.462093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:45665 (local address 127.18.45.193:38475)
0806 22:45:00.462272 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:00.462322 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:00.462917 (+ 595us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:00.463268 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:00.463272 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:00.463592 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:00.464187 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:00.464193 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:00.465020 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:00.465021 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:00.465921 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:00.465923 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:00.466328 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:00.466329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:00.466455 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:00.467101 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:00.467111 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:00.468840 (+ 1729us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:45:00.470785 (+ 1945us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:00.470793 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:00.470796 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:00.471260 (+ 464us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:00.471643 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:00.471644 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:00.471645 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:00.471792 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:00.472191 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:45:00.472197 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:45:00.472342 (+ 145us) client_negotiation.cc:769] Sending connection context
0806 22:45:00.472436 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 22:45:00.472605 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":94,"threads_started":1}
I20260806 22:45:00.473774 24769 heartbeater.cc:347] Connected to a master server at 127.18.45.254:45665
I20260806 22:45:00.473918 24769 heartbeater.cc:464] Registering TS with master...
I20260806 22:45:00.474138 24769 heartbeater.cc:511] Master 127.18.45.254:45665 requested a full tablet report, sending...
I20260806 22:45:00.474828 24498 ts_manager.cc:194] Registered new tserver with Master: e8d589e6c8a6423da76d4b3080cb651a (127.18.45.193:37491)
I20260806 22:45:00.475543 24498 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.45.193@KRBTEST.COM'} at 127.18.45.193:38475
WARNING: no policy specified for kudu/127.18.45.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab.
Entry for principal kudu/127.18.45.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab.
I20260806 22:45:00.507776 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.194:0
--local_ip_for_outbound_sockets=127.18.45.194
--webserver_interface=127.18.45.194
--webserver_port=0
--tserver_master_addrs=127.18.45.254:45665
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab
--principal=kudu/127.18.45.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:38945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:45:00.628669 24775 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:45:00.629046 24775 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:45:00.629091 24775 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:45:00.630436 24775 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:45:00.630488 24775 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:45:00.630518 24775 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.194
I20260806 22:45:00.632334 24775 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:38945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab
--principal=kudu/127.18.45.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.45.194
--webserver_port=0
--tserver_master_addrs=127.18.45.254:45665
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24775
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.194
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:45:00.632681 24775 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:45:00.633044 24775 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:45:00.639106 24789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:00.638989 24786 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:00.638967 24795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:00.640434 24775 server_base.cc:1034] running on GCE node
I20260806 22:45:00.640657 24775 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:45:00.640978 24775 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:45:00.642199 24775 hybrid_clock.cc:648] HybridClock initialized: now 1786056300642182 us; error 30 us; skew 500 ppm
Aug 06 22:45:00 dist-test-slave-h3ft krb5kdc[24435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056300, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:45:00.645720 24775 init.cc:377] Logged in from keytab as kudu/127.18.45.194@KRBTEST.COM (short username kudu)
I20260806 22:45:00.646273 24775 fs_manager.cc:359] Metadata directory not provided
I20260806 22:45:00.646366 24775 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:45:00.646569 24775 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:45:00.648164 24775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/minicluster-data/ts-1/data/instance:
uuid: "fd13146fdf564ddc93c20f4509b89581"
format_stamp: "Formatted at 2026-08-06 22:45:00 on dist-test-slave-h3ft"
server_key: "dc5d0877af227c56652937e3825df0c5"
server_key_iv: "c8a57b6c786f39c11246e0b600e2133c"
server_key_version: "encryptionkey@0"
I20260806 22:45:00.648650 24775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/minicluster-data/ts-1/wal/instance:
uuid: "fd13146fdf564ddc93c20f4509b89581"
format_stamp: "Formatted at 2026-08-06 22:45:00 on dist-test-slave-h3ft"
server_key: "dc5d0877af227c56652937e3825df0c5"
server_key_iv: "c8a57b6c786f39c11246e0b600e2133c"
server_key_version: "encryptionkey@0"
I20260806 22:45:00.650405 24775 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260806 22:45:00.651432 24805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:00.651670 24775 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:45:00.651793 24775 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/minicluster-data/ts-1/wal
uuid: "fd13146fdf564ddc93c20f4509b89581"
format_stamp: "Formatted at 2026-08-06 22:45:00 on dist-test-slave-h3ft"
server_key: "dc5d0877af227c56652937e3825df0c5"
server_key_iv: "c8a57b6c786f39c11246e0b600e2133c"
server_key_version: "encryptionkey@0"
I20260806 22:45:00.651888 24775 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:45:00.676033 24775 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:45:00.679553 24775 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:45:00.679760 24775 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:45:00.680130 24775 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:45:00.680583 24775 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:45:00.680622 24775 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:00.680651 24775 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:45:00.680665 24775 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:00.688534 24775 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.194:46189
I20260806 22:45:00.688575 24971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.194:46189 every 8 connection(s)
I20260806 22:45:00.688933 24775 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
I20260806 22:45:00.696548 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 24775
I20260806 22:45:00.696717 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/minicluster-data/ts-1/wal/instance
Aug 06 22:45:00 dist-test-slave-h3ft krb5kdc[24435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056300, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.194@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:45:00.696982 18615 external_mini_cluster.cc:1428] Setting key f677225d8508567c4f031dc9a877daef
I20260806 22:45:00.701418 24549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:00.689873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:41899 (local address 127.18.45.254:45665)
0806 22:45:00.689985 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:00.689987 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:00.690194 (+ 207us) server_negotiation.cc:407] Connection header received
0806 22:45:00.690833 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:00.690835 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:00.690851 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:00.690972 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:00.692834 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:00.693419 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:00.694549 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:00.694768 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:00.694843 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:00.697622 (+ 2779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:00.697623 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:00.697623 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:00.697631 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:00.699555 (+ 1924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:00.700001 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:00.700002 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:00.700002 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:00.700081 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:00.700411 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:00.700412 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:00.700413 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:00.700600 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:00.700661 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:00.701231 (+ 570us) server_negotiation.cc:299] Negotiation successful
0806 22:45:00.701337 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 22:45:00.701656 24977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:00.689787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:45665 (local address 127.18.45.194:41899)
0806 22:45:00.690026 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:00.690079 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:00.690710 (+ 631us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:00.691153 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:00.691160 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:00.691712 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:00.692698 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:00.692707 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:00.693529 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:00.693529 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:00.694444 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:00.694447 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:00.694858 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:00.694859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:00.694992 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:00.695687 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:00.695697 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:00.697527 (+ 1830us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:45:00.699654 (+ 2127us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:00.699662 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:00.699664 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:00.699922 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:00.700152 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:00.700153 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:00.700153 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:00.700325 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:00.700672 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:45:00.700680 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:45:00.700884 (+ 204us) client_negotiation.cc:769] Sending connection context
0806 22:45:00.701030 (+ 146us) client_negotiation.cc:240] Negotiation successful
0806 22:45:00.701249 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":125,"threads_started":1}
I20260806 22:45:00.702338 24975 heartbeater.cc:347] Connected to a master server at 127.18.45.254:45665
I20260806 22:45:00.702497 24975 heartbeater.cc:464] Registering TS with master...
I20260806 22:45:00.702708 24975 heartbeater.cc:511] Master 127.18.45.254:45665 requested a full tablet report, sending...
I20260806 22:45:00.703298 24498 ts_manager.cc:194] Registered new tserver with Master: fd13146fdf564ddc93c20f4509b89581 (127.18.45.194:46189)
I20260806 22:45:00.703763 24498 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.45.194@KRBTEST.COM'} at 127.18.45.194:41899
WARNING: no policy specified for kudu/127.18.45.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab.
Entry for principal kudu/127.18.45.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab.
I20260806 22:45:00.732218 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.195:0
--local_ip_for_outbound_sockets=127.18.45.195
--webserver_interface=127.18.45.195
--webserver_port=0
--tserver_master_addrs=127.18.45.254:45665
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab
--principal=kudu/127.18.45.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:38945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:45:00.836853 24982 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:45:00.837146 24982 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:45:00.837188 24982 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:45:00.838438 24982 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:45:00.838510 24982 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:45:00.838557 24982 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.195
I20260806 22:45:00.840404 24982 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:38945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab
--principal=kudu/127.18.45.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.45.195
--webserver_port=0
--tserver_master_addrs=127.18.45.254:45665
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24982
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.195
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:45:00.840721 24982 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:45:00.841154 24982 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:45:00.849797 24995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:00.849797 24990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:00.850395 24982 server_base.cc:1034] running on GCE node
W20260806 22:45:00.851315 24997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:00.851656 24982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:45:00.852068 24982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:45:00.853257 24982 hybrid_clock.cc:648] HybridClock initialized: now 1786056300853215 us; error 63 us; skew 500 ppm
Aug 06 22:45:00 dist-test-slave-h3ft krb5kdc[24435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056300, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:45:00.857191 24982 init.cc:377] Logged in from keytab as kudu/127.18.45.195@KRBTEST.COM (short username kudu)
I20260806 22:45:00.857705 24982 fs_manager.cc:359] Metadata directory not provided
I20260806 22:45:00.857765 24982 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:45:00.858021 24982 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:45:00.859516 24982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/minicluster-data/ts-2/data/instance:
uuid: "0a55364713bd4ae9b31710ad0326e296"
format_stamp: "Formatted at 2026-08-06 22:45:00 on dist-test-slave-h3ft"
server_key: "7d282496b3ab7e7c41198c66200ea59f"
server_key_iv: "4a7425a61a4d66d89fb36dbf0988f9da"
server_key_version: "encryptionkey@0"
I20260806 22:45:00.860081 24982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/minicluster-data/ts-2/wal/instance:
uuid: "0a55364713bd4ae9b31710ad0326e296"
format_stamp: "Formatted at 2026-08-06 22:45:00 on dist-test-slave-h3ft"
server_key: "7d282496b3ab7e7c41198c66200ea59f"
server_key_iv: "4a7425a61a4d66d89fb36dbf0988f9da"
server_key_version: "encryptionkey@0"
I20260806 22:45:00.862001 24982 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260806 22:45:00.862979 25004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:00.863286 24982 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:45:00.863408 24982 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/minicluster-data/ts-2/wal
uuid: "0a55364713bd4ae9b31710ad0326e296"
format_stamp: "Formatted at 2026-08-06 22:45:00 on dist-test-slave-h3ft"
server_key: "7d282496b3ab7e7c41198c66200ea59f"
server_key_iv: "4a7425a61a4d66d89fb36dbf0988f9da"
server_key_version: "encryptionkey@0"
I20260806 22:45:00.863509 24982 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:45:00.882881 24982 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:45:00.885550 24982 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:45:00.885732 24982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:45:00.886068 24982 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:45:00.886508 24982 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:45:00.886546 24982 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:00.886595 24982 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:45:00.886626 24982 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:00.894259 24982 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.195:43841
I20260806 22:45:00.894311 25159 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.195:43841 every 8 connection(s)
I20260806 22:45:00.894654 24982 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
I20260806 22:45:00.900815 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 24982
I20260806 22:45:00.900925 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/minicluster-data/ts-2/wal/instance
I20260806 22:45:00.901186 18615 external_mini_cluster.cc:1428] Setting key 57020ebc998154566b33a64c0a248fb5
Aug 06 22:45:00 dist-test-slave-h3ft krb5kdc[24435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056300, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.195@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:45:00.906219 24549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:00.895700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.195:34823 (local address 127.18.45.254:45665)
0806 22:45:00.895891 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:00.895893 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:00.896067 (+ 174us) server_negotiation.cc:407] Connection header received
0806 22:45:00.896663 (+ 596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:00.896664 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:00.896678 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:00.896750 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:00.897984 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:00.898584 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:00.899812 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:00.900076 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:00.900142 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:00.902963 (+ 2821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:00.902965 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:00.902965 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:00.902976 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:00.904595 (+ 1619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:00.905076 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:00.905078 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:00.905079 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:00.905150 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:00.905444 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:00.905445 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:00.905446 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:00.905594 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:00.905681 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:00.906009 (+ 328us) server_negotiation.cc:299] Negotiation successful
0806 22:45:00.906111 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153}
I20260806 22:45:00.906461 25162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:00.895659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:45665 (local address 127.18.45.195:34823)
0806 22:45:00.895866 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:00.895919 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:00.896544 (+ 625us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:00.896851 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:00.896856 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:00.897201 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:00.897857 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:00.897865 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:00.898691 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:00.898692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:00.899714 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:00.899717 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:00.900177 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:00.900178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:00.900315 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:00.901129 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:00.901140 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:00.902854 (+ 1714us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:45:00.904706 (+ 1852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:00.904717 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:00.904720 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:00.904981 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:00.905225 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:00.905225 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:00.905226 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:00.905358 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:00.905671 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:45:00.905681 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:45:00.905856 (+ 175us) client_negotiation.cc:769] Sending connection context
0806 22:45:00.905984 (+ 128us) client_negotiation.cc:240] Negotiation successful
0806 22:45:00.906173 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":105,"threads_started":1}
I20260806 22:45:00.907109 25160 heartbeater.cc:347] Connected to a master server at 127.18.45.254:45665
I20260806 22:45:00.907234 25160 heartbeater.cc:464] Registering TS with master...
I20260806 22:45:00.907387 25160 heartbeater.cc:511] Master 127.18.45.254:45665 requested a full tablet report, sending...
I20260806 22:45:00.907861 24498 ts_manager.cc:194] Registered new tserver with Master: 0a55364713bd4ae9b31710ad0326e296 (127.18.45.195:43841)
I20260806 22:45:00.908318 24498 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.45.195@KRBTEST.COM'} at 127.18.45.195:34823
I20260806 22:45:00.913384 18615 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:45:00.922158 24549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:00.915095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38224 (local address 127.18.45.254:45665)
0806 22:45:00.915166 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:00.915173 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:00.915348 (+ 175us) server_negotiation.cc:407] Connection header received
0806 22:45:00.915379 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:00.915380 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:00.915393 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:00.915486 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:00.916181 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:00.916722 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:00.917485 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:00.917703 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:00.917763 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:00.919187 (+ 1424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:00.919189 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:00.919189 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:00.919197 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:00.920930 (+ 1733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:00.921304 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:00.921305 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:00.921305 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:00.921370 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:00.921570 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:00.921571 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:00.921571 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:00.921753 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:00.921846 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:00.921991 (+ 145us) server_negotiation.cc:299] Negotiation successful
0806 22:45:00.922072 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 22:45:00.924162 24498 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38224:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:45:00.926222 24498 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:45:00.934703 25171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:00.929765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.194:46189 (local address 127.0.0.1:50978)
0806 22:45:00.930032 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:00.930042 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:00.930093 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:00.930782 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:00.930783 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:00.930786 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:45:00.930836 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:00.930837 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:00.932519 (+ 1682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:00.932520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:00.933401 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:00.933404 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:00.934263 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:00.934264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:00.934362 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:00.934381 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 22:45:00.934499 (+ 118us) client_negotiation.cc:240] Negotiation successful
0806 22:45:00.934580 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":72,"threads_started":1}
I20260806 22:45:00.934849 25172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:00.929521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.193:37491 (local address 127.0.0.1:56064)
0806 22:45:00.929770 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:00.929826 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:00.929906 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:00.930883 (+ 977us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:00.930884 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:00.930887 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:45:00.930953 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:00.930955 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:00.931950 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:00.931951 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:00.933465 (+ 1514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:00.933468 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:00.934514 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:00.934515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:00.934586 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:00.934591 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:45:00.934627 (+ 36us) client_negotiation.cc:240] Negotiation successful
0806 22:45:00.934688 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":87,"threads_started":1}
I20260806 22:45:00.935034 25175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:00.929532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.195:43841 (local address 127.0.0.1:53402)
0806 22:45:00.930029 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:00.930040 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:00.930093 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:00.930541 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:00.930545 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:00.930548 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:45:00.930629 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:00.930631 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:00.932344 (+ 1713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:00.932345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:00.933491 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:00.933493 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:00.934741 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:00.934742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:00.934818 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:00.934822 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:45:00.934882 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 22:45:00.934941 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"mutex_wait_us":32,"thread_start_us":90,"threads_started":1}
I20260806 22:45:00.935086 25176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:00.930056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50978 (local address 127.18.45.194:46189)
0806 22:45:00.930298 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:00.930304 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:00.930317 (+ 13us) server_negotiation.cc:407] Connection header received
0806 22:45:00.930369 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:00.930371 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:00.930391 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:00.930496 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:45:00.930985 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:00.932399 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:00.933516 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:00.934147 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:00.934250 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:00.934333 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:00.934894 (+ 561us) server_negotiation.cc:299] Negotiation successful
0806 22:45:00.934953 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":127,"threads_started":1}
I20260806 22:45:00.935403 25174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:00.929687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53402 (local address 127.18.45.195:43841)
0806 22:45:00.930298 (+ 611us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:00.930304 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:00.930317 (+ 13us) server_negotiation.cc:407] Connection header received
0806 22:45:00.930369 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:00.930371 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:00.930391 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:00.930496 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:45:00.930759 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:00.932183 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:00.933598 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:00.934635 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:00.934731 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:00.934796 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:00.935244 (+ 448us) server_negotiation.cc:299] Negotiation successful
0806 22:45:00.935305 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":159,"threads_started":1}
I20260806 22:45:00.935427 25173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:00.929701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56064 (local address 127.18.45.193:37491)
0806 22:45:00.930707 (+ 1006us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:00.930711 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:00.930720 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:45:00.930759 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:00.930761 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:00.930774 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:00.930870 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:45:00.931078 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:00.931826 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:00.933598 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:00.934406 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:00.934757 (+ 351us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:00.935219 (+ 462us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:00.935247 (+ 28us) server_negotiation.cc:299] Negotiation successful
0806 22:45:00.935301 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":965,"thread_start_us":113,"threads_started":1}
I20260806 22:45:00.936090 25068 tablet_service.cc:1467] Processing CreateTablet for tablet bfee928f7afa468c822d99451379d867 (DEFAULT_TABLE table=test-table [id=6bf72dc0eec2411b8aa2947eaa7ee383]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:45:00.936049 24874 tablet_service.cc:1467] Processing CreateTablet for tablet bfee928f7afa468c822d99451379d867 (DEFAULT_TABLE table=test-table [id=6bf72dc0eec2411b8aa2947eaa7ee383]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:45:00.936141 24675 tablet_service.cc:1467] Processing CreateTablet for tablet bfee928f7afa468c822d99451379d867 (DEFAULT_TABLE table=test-table [id=6bf72dc0eec2411b8aa2947eaa7ee383]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:45:00.936556 24675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfee928f7afa468c822d99451379d867. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:45:00.936551 25068 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfee928f7afa468c822d99451379d867. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:45:00.936568 24874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfee928f7afa468c822d99451379d867. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:45:00.939929 25177 tablet_bootstrap.cc:492] T bfee928f7afa468c822d99451379d867 P e8d589e6c8a6423da76d4b3080cb651a: Bootstrap starting.
I20260806 22:45:00.940783 25177 tablet_bootstrap.cc:654] T bfee928f7afa468c822d99451379d867 P e8d589e6c8a6423da76d4b3080cb651a: Neither blocks nor log segments found. Creating new log.
I20260806 22:45:00.941286 25177 log.cc:824] T bfee928f7afa468c822d99451379d867 P e8d589e6c8a6423da76d4b3080cb651a: Log is configured to *not* fsync() on all Append() calls
I20260806 22:45:00.941484 25180 tablet_bootstrap.cc:492] T bfee928f7afa468c822d99451379d867 P 0a55364713bd4ae9b31710ad0326e296: Bootstrap starting.
I20260806 22:45:00.941625 25179 tablet_bootstrap.cc:492] T bfee928f7afa468c822d99451379d867 P fd13146fdf564ddc93c20f4509b89581: Bootstrap starting.
I20260806 22:45:00.942204 25177 tablet_bootstrap.cc:492] T bfee928f7afa468c822d99451379d867 P e8d589e6c8a6423da76d4b3080cb651a: No bootstrap required, opened a new log
I20260806 22:45:00.942301 25177 ts_tablet_manager.cc:1397] T bfee928f7afa468c822d99451379d867 P e8d589e6c8a6423da76d4b3080cb651a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:45:00.942412 25179 tablet_bootstrap.cc:654] T bfee928f7afa468c822d99451379d867 P fd13146fdf564ddc93c20f4509b89581: Neither blocks nor log segments found. Creating new log.
I20260806 22:45:00.942423 25180 tablet_bootstrap.cc:654] T bfee928f7afa468c822d99451379d867 P 0a55364713bd4ae9b31710ad0326e296: Neither blocks nor log segments found. Creating new log.
I20260806 22:45:00.943290 25180 log.cc:824] T bfee928f7afa468c822d99451379d867 P 0a55364713bd4ae9b31710ad0326e296: Log is configured to *not* fsync() on all Append() calls
I20260806 22:45:00.943516 25179 log.cc:824] T bfee928f7afa468c822d99451379d867 P fd13146fdf564ddc93c20f4509b89581: Log is configured to *not* fsync() on all Append() calls
I20260806 22:45:00.944330 25180 tablet_bootstrap.cc:492] T bfee928f7afa468c822d99451379d867 P 0a55364713bd4ae9b31710ad0326e296: No bootstrap required, opened a new log
I20260806 22:45:00.944394 25180 ts_tablet_manager.cc:1397] T bfee928f7afa468c822d99451379d867 P 0a55364713bd4ae9b31710ad0326e296: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:45:00.944761 25177 raft_consensus.cc:348] T bfee928f7afa468c822d99451379d867 P e8d589e6c8a6423da76d4b3080cb651a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8d589e6c8a6423da76d4b3080cb651a" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 37491 } } peers { permanent_uuid: "0a55364713bd4ae9b31710ad0326e296" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 43841 } } peers { permanent_uuid: "fd13146fdf564ddc93c20f4509b89581" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 46189 } }
I20260806 22:45:00.944949 25177 raft_consensus.cc:374] T bfee928f7afa468c822d99451379d867 P e8d589e6c8a6423da76d4b3080cb651a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:45:00.945047 25177 raft_consensus.cc:729] T bfee928f7afa468c822d99451379d867 P e8d589e6c8a6423da76d4b3080cb651a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8d589e6c8a6423da76d4b3080cb651a, State: Initialized, Role: FOLLOWER
I20260806 22:45:00.945225 25177 consensus_queue.cc:260] T bfee928f7afa468c822d99451379d867 P e8d589e6c8a6423da76d4b3080cb651a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8d589e6c8a6423da76d4b3080cb651a" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 37491 } } peers { permanent_uuid: "0a55364713bd4ae9b31710ad0326e296" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 43841 } } peers { permanent_uuid: "fd13146fdf564ddc93c20f4509b89581" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 46189 } }
I20260806 22:45:00.945578 25177 ts_tablet_manager.cc:1428] T bfee928f7afa468c822d99451379d867 P e8d589e6c8a6423da76d4b3080cb651a: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260806 22:45:00.945617 24769 heartbeater.cc:503] Master 127.18.45.254:45665 was elected leader, sending a full tablet report...
I20260806 22:45:00.946698 25180 raft_consensus.cc:348] T bfee928f7afa468c822d99451379d867 P 0a55364713bd4ae9b31710ad0326e296 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8d589e6c8a6423da76d4b3080cb651a" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 37491 } } peers { permanent_uuid: "0a55364713bd4ae9b31710ad0326e296" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 43841 } } peers { permanent_uuid: "fd13146fdf564ddc93c20f4509b89581" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 46189 } }
I20260806 22:45:00.946897 25180 raft_consensus.cc:374] T bfee928f7afa468c822d99451379d867 P 0a55364713bd4ae9b31710ad0326e296 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:45:00.946965 25180 raft_consensus.cc:729] T bfee928f7afa468c822d99451379d867 P 0a55364713bd4ae9b31710ad0326e296 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a55364713bd4ae9b31710ad0326e296, State: Initialized, Role: FOLLOWER
I20260806 22:45:00.947211 25180 consensus_queue.cc:260] T bfee928f7afa468c822d99451379d867 P 0a55364713bd4ae9b31710ad0326e296 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8d589e6c8a6423da76d4b3080cb651a" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 37491 } } peers { permanent_uuid: "0a55364713bd4ae9b31710ad0326e296" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 43841 } } peers { permanent_uuid: "fd13146fdf564ddc93c20f4509b89581" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 46189 } }
I20260806 22:45:00.947418 25179 tablet_bootstrap.cc:492] T bfee928f7afa468c822d99451379d867 P fd13146fdf564ddc93c20f4509b89581: No bootstrap required, opened a new log
I20260806 22:45:00.947520 25179 ts_tablet_manager.cc:1397] T bfee928f7afa468c822d99451379d867 P fd13146fdf564ddc93c20f4509b89581: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260806 22:45:00.947604 25180 ts_tablet_manager.cc:1428] T bfee928f7afa468c822d99451379d867 P 0a55364713bd4ae9b31710ad0326e296: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:45:00.947755 25160 heartbeater.cc:503] Master 127.18.45.254:45665 was elected leader, sending a full tablet report...
I20260806 22:45:00.949575 25179 raft_consensus.cc:348] T bfee928f7afa468c822d99451379d867 P fd13146fdf564ddc93c20f4509b89581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8d589e6c8a6423da76d4b3080cb651a" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 37491 } } peers { permanent_uuid: "0a55364713bd4ae9b31710ad0326e296" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 43841 } } peers { permanent_uuid: "fd13146fdf564ddc93c20f4509b89581" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 46189 } }
I20260806 22:45:00.949750 25179 raft_consensus.cc:374] T bfee928f7afa468c822d99451379d867 P fd13146fdf564ddc93c20f4509b89581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:45:00.949800 25179 raft_consensus.cc:729] T bfee928f7afa468c822d99451379d867 P fd13146fdf564ddc93c20f4509b89581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd13146fdf564ddc93c20f4509b89581, State: Initialized, Role: FOLLOWER
I20260806 22:45:00.949926 25179 consensus_queue.cc:260] T bfee928f7afa468c822d99451379d867 P fd13146fdf564ddc93c20f4509b89581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8d589e6c8a6423da76d4b3080cb651a" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 37491 } } peers { permanent_uuid: "0a55364713bd4ae9b31710ad0326e296" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 43841 } } peers { permanent_uuid: "fd13146fdf564ddc93c20f4509b89581" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 46189 } }
I20260806 22:45:00.950253 25179 ts_tablet_manager.cc:1428] T bfee928f7afa468c822d99451379d867 P fd13146fdf564ddc93c20f4509b89581: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:45:00.950335 24975 heartbeater.cc:503] Master 127.18.45.254:45665 was elected leader, sending a full tablet report...
W20260806 22:45:00.962066 24770 tablet.cc:2369] T bfee928f7afa468c822d99451379d867 P e8d589e6c8a6423da76d4b3080cb651a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:45:00.978654 25187 raft_consensus.cc:482] T bfee928f7afa468c822d99451379d867 P fd13146fdf564ddc93c20f4509b89581 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:45:00.978883 25187 raft_consensus.cc:504] T bfee928f7afa468c822d99451379d867 P fd13146fdf564ddc93c20f4509b89581 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8d589e6c8a6423da76d4b3080cb651a" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 37491 } } peers { permanent_uuid: "0a55364713bd4ae9b31710ad0326e296" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 43841 } } peers { permanent_uuid: "fd13146fdf564ddc93c20f4509b89581" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 46189 } }
I20260806 22:45:00.979411 25187 leader_election.cc:288] T bfee928f7afa468c822d99451379d867 P fd13146fdf564ddc93c20f4509b89581 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8d589e6c8a6423da76d4b3080cb651a (127.18.45.193:37491), 0a55364713bd4ae9b31710ad0326e296 (127.18.45.195:43841)
I20260806 22:45:00.982820 25174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:00.979756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:42017 (local address 127.18.45.195:43841)
0806 22:45:00.979813 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:00.979816 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:00.980003 (+ 187us) server_negotiation.cc:407] Connection header received
0806 22:45:00.980108 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:00.980110 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:00.980124 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:00.980191 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:45:00.980406 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:00.980932 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:00.981965 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:00.982403 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:00.982503 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:00.982514 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:00.982696 (+ 182us) server_negotiation.cc:299] Negotiation successful
0806 22:45:00.982743 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 22:45:00.982817 24977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:00.979675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.193:37491 (local address 127.18.45.194:60017)
0806 22:45:00.979749 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:00.979764 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:00.979819 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:00.980039 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:00.980041 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:00.980043 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:45:00.980158 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:00.980161 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:00.980935 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:00.980937 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:00.981863 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:00.981866 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:00.982474 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:00.982475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:00.982586 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:00.982590 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:45:00.982635 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 22:45:00.982704 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46,"mutex_wait_us":32}
I20260806 22:45:00.982815 25173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:00.979756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:60017 (local address 127.18.45.193:37491)
0806 22:45:00.979813 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:00.979816 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:00.979828 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:45:00.979899 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:00.979900 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:00.979914 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:00.980029 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:45:00.980242 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:00.980839 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:00.981977 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:00.982382 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:00.982442 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:00.982451 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:00.982665 (+ 214us) server_negotiation.cc:299] Negotiation successful
0806 22:45:00.982718 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 22:45:00.982827 25189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:00.979676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.195:43841 (local address 127.18.45.194:42017)
0806 22:45:00.979928 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:00.979939 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:00.980035 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:00.980224 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:00.980225 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:00.980227 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:45:00.980325 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:00.980326 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:00.981031 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:00.981032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:00.981862 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:00.981864 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:00.982528 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:00.982529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:00.982597 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:00.982601 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:45:00.982675 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 22:45:00.982720 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"mutex_wait_us":29,"thread_start_us":119,"threads_started":1}
I20260806 22:45:00.983322 24706 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfee928f7afa468c822d99451379d867" candidate_uuid: "fd13146fdf564ddc93c20f4509b89581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8d589e6c8a6423da76d4b3080cb651a" is_pre_election: true
I20260806 22:45:00.983587 24706 raft_consensus.cc:2454] T bfee928f7afa468c822d99451379d867 P e8d589e6c8a6423da76d4b3080cb651a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd13146fdf564ddc93c20f4509b89581 in term 0.
I20260806 22:45:00.983577 25097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfee928f7afa468c822d99451379d867" candidate_uuid: "fd13146fdf564ddc93c20f4509b89581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a55364713bd4ae9b31710ad0326e296" is_pre_election: true
I20260806 22:45:00.983779 25097 raft_consensus.cc:2454] T bfee928f7afa468c822d99451379d867 P 0a55364713bd4ae9b31710ad0326e296 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd13146fdf564ddc93c20f4509b89581 in term 0.
I20260806 22:45:00.984053 24807 leader_election.cc:302] T bfee928f7afa468c822d99451379d867 P fd13146fdf564ddc93c20f4509b89581 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a55364713bd4ae9b31710ad0326e296, fd13146fdf564ddc93c20f4509b89581; no voters:
I20260806 22:45:00.984215 25187 raft_consensus.cc:2781] T bfee928f7afa468c822d99451379d867 P fd13146fdf564ddc93c20f4509b89581 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:45:00.984292 25187 raft_consensus.cc:482] T bfee928f7afa468c822d99451379d867 P fd13146fdf564ddc93c20f4509b89581 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:45:00.984319 25187 raft_consensus.cc:3037] T bfee928f7afa468c822d99451379d867 P fd13146fdf564ddc93c20f4509b89581 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:45:00.985245 25187 raft_consensus.cc:504] T bfee928f7afa468c822d99451379d867 P fd13146fdf564ddc93c20f4509b89581 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8d589e6c8a6423da76d4b3080cb651a" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 37491 } } peers { permanent_uuid: "0a55364713bd4ae9b31710ad0326e296" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 43841 } } peers { permanent_uuid: "fd13146fdf564ddc93c20f4509b89581" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 46189 } }
I20260806 22:45:00.985498 25187 leader_election.cc:288] T bfee928f7afa468c822d99451379d867 P fd13146fdf564ddc93c20f4509b89581 [CANDIDATE]: Term 1 election: Requested vote from peers e8d589e6c8a6423da76d4b3080cb651a (127.18.45.193:37491), 0a55364713bd4ae9b31710ad0326e296 (127.18.45.195:43841)
I20260806 22:45:00.985589 25097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfee928f7afa468c822d99451379d867" candidate_uuid: "fd13146fdf564ddc93c20f4509b89581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a55364713bd4ae9b31710ad0326e296"
I20260806 22:45:00.985621 24706 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfee928f7afa468c822d99451379d867" candidate_uuid: "fd13146fdf564ddc93c20f4509b89581" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8d589e6c8a6423da76d4b3080cb651a"
I20260806 22:45:00.985685 25097 raft_consensus.cc:3037] T bfee928f7afa468c822d99451379d867 P 0a55364713bd4ae9b31710ad0326e296 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:45:00.985702 24706 raft_consensus.cc:3037] T bfee928f7afa468c822d99451379d867 P e8d589e6c8a6423da76d4b3080cb651a [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:45:00.986586 24706 raft_consensus.cc:2454] T bfee928f7afa468c822d99451379d867 P e8d589e6c8a6423da76d4b3080cb651a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd13146fdf564ddc93c20f4509b89581 in term 1.
I20260806 22:45:00.986662 25097 raft_consensus.cc:2454] T bfee928f7afa468c822d99451379d867 P 0a55364713bd4ae9b31710ad0326e296 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd13146fdf564ddc93c20f4509b89581 in term 1.
I20260806 22:45:00.986786 24811 leader_election.cc:302] T bfee928f7afa468c822d99451379d867 P fd13146fdf564ddc93c20f4509b89581 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e8d589e6c8a6423da76d4b3080cb651a, fd13146fdf564ddc93c20f4509b89581; no voters:
I20260806 22:45:00.986908 25187 raft_consensus.cc:2781] T bfee928f7afa468c822d99451379d867 P fd13146fdf564ddc93c20f4509b89581 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:45:00.987161 25187 raft_consensus.cc:686] T bfee928f7afa468c822d99451379d867 P fd13146fdf564ddc93c20f4509b89581 [term 1 LEADER]: Becoming Leader. State: Replica: fd13146fdf564ddc93c20f4509b89581, State: Running, Role: LEADER
I20260806 22:45:00.987329 25187 consensus_queue.cc:237] T bfee928f7afa468c822d99451379d867 P fd13146fdf564ddc93c20f4509b89581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8d589e6c8a6423da76d4b3080cb651a" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 37491 } } peers { permanent_uuid: "0a55364713bd4ae9b31710ad0326e296" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 43841 } } peers { permanent_uuid: "fd13146fdf564ddc93c20f4509b89581" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 46189 } }
I20260806 22:45:00.988576 24498 catalog_manager.cc:5495] T bfee928f7afa468c822d99451379d867 P fd13146fdf564ddc93c20f4509b89581 reported cstate change: term changed from 0 to 1, leader changed from <none> to fd13146fdf564ddc93c20f4509b89581 (127.18.45.194). New cstate: current_term: 1 leader_uuid: "fd13146fdf564ddc93c20f4509b89581" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8d589e6c8a6423da76d4b3080cb651a" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 37491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a55364713bd4ae9b31710ad0326e296" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 43841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd13146fdf564ddc93c20f4509b89581" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 46189 } health_report { overall_health: HEALTHY } } }
I20260806 22:45:00.995252 25176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:00.992376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50992 (local address 127.18.45.194:46189)
0806 22:45:00.992437 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:00.992439 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:00.992448 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:45:00.992526 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:00.992527 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:00.992548 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:00.992637 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:45:00.992841 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:00.993420 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:00.994349 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:00.994558 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:00.994623 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:00.994821 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:45:00.994946 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:45:00.995094 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:00.995132 (+ 38us) server_negotiation.cc:299] Negotiation successful
0806 22:45:00.995156 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 22:45:00.997519 25097 raft_consensus.cc:1261] T bfee928f7afa468c822d99451379d867 P 0a55364713bd4ae9b31710ad0326e296 [term 1 FOLLOWER]: Refusing update from remote peer fd13146fdf564ddc93c20f4509b89581: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:45:00.997516 24706 raft_consensus.cc:1261] T bfee928f7afa468c822d99451379d867 P e8d589e6c8a6423da76d4b3080cb651a [term 1 FOLLOWER]: Refusing update from remote peer fd13146fdf564ddc93c20f4509b89581: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:45:00.997958 25190 consensus_queue.cc:1035] T bfee928f7afa468c822d99451379d867 P fd13146fdf564ddc93c20f4509b89581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a55364713bd4ae9b31710ad0326e296" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 43841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:45:00.998126 25187 consensus_queue.cc:1035] T bfee928f7afa468c822d99451379d867 P fd13146fdf564ddc93c20f4509b89581 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8d589e6c8a6423da76d4b3080cb651a" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 37491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:45:01.001881 25197 mvcc.cc:205] Tried to move back new op lower bound from 7315686608882475008 to 7315686608845164544. Current Snapshot: MvccSnapshot[applied={T|T < 7315686608882475008}]
I20260806 22:45:01.002293 25198 mvcc.cc:205] Tried to move back new op lower bound from 7315686608882475008 to 7315686608845164544. Current Snapshot: MvccSnapshot[applied={T|T < 7315686608882475008}]
I20260806 22:45:01.004398 24494 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38224:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:45:01.004550 24494 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38224:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:45:01.005592 24494 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2289b5a3377b434e9fda5c21f411a194: Sending DeleteTablet for 3 replicas of tablet bfee928f7afa468c822d99451379d867
I20260806 22:45:01.006073 24874 tablet_service.cc:1514] Processing DeleteTablet for tablet bfee928f7afa468c822d99451379d867 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:45:01 UTC) from {username='kudu', principal='kudu/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:50978
I20260806 22:45:01.006105 25068 tablet_service.cc:1514] Processing DeleteTablet for tablet bfee928f7afa468c822d99451379d867 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:45:01 UTC) from {username='kudu', principal='kudu/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:53402
I20260806 22:45:01.006173 24675 tablet_service.cc:1514] Processing DeleteTablet for tablet bfee928f7afa468c822d99451379d867 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:45:01 UTC) from {username='kudu', principal='kudu/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:56064
I20260806 22:45:01.006420 25214 tablet_replica.cc:330] T bfee928f7afa468c822d99451379d867 P 0a55364713bd4ae9b31710ad0326e296: stopping tablet replica
I20260806 22:45:01.006556 25213 tablet_replica.cc:330] T bfee928f7afa468c822d99451379d867 P fd13146fdf564ddc93c20f4509b89581: stopping tablet replica
I20260806 22:45:01.006637 25216 tablet_replica.cc:330] T bfee928f7afa468c822d99451379d867 P e8d589e6c8a6423da76d4b3080cb651a: stopping tablet replica
I20260806 22:45:01.006695 25213 raft_consensus.cc:2229] T bfee928f7afa468c822d99451379d867 P fd13146fdf564ddc93c20f4509b89581 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:45:01.006775 25216 raft_consensus.cc:2229] T bfee928f7afa468c822d99451379d867 P e8d589e6c8a6423da76d4b3080cb651a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:45:01.006910 25216 raft_consensus.cc:2258] T bfee928f7afa468c822d99451379d867 P e8d589e6c8a6423da76d4b3080cb651a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:45:01.006917 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 24580
I20260806 22:45:01.006935 25213 raft_consensus.cc:2258] T bfee928f7afa468c822d99451379d867 P fd13146fdf564ddc93c20f4509b89581 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:45:01.013916 25213 ts_tablet_manager.cc:1905] T bfee928f7afa468c822d99451379d867 P fd13146fdf564ddc93c20f4509b89581: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:45:01.015667 25214 raft_consensus.cc:2229] T bfee928f7afa468c822d99451379d867 P 0a55364713bd4ae9b31710ad0326e296 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:45:01.015981 25214 raft_consensus.cc:2258] T bfee928f7afa468c822d99451379d867 P 0a55364713bd4ae9b31710ad0326e296 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:45:01.016135 25213 ts_tablet_manager.cc:1918] T bfee928f7afa468c822d99451379d867 P fd13146fdf564ddc93c20f4509b89581: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:45:01.016228 25213 log.cc:1196] T bfee928f7afa468c822d99451379d867 P fd13146fdf564ddc93c20f4509b89581: Deleting WAL directory at /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/minicluster-data/ts-1/wal/wals/bfee928f7afa468c822d99451379d867
I20260806 22:45:01.016821 25213 ts_tablet_manager.cc:1939] T bfee928f7afa468c822d99451379d867 P fd13146fdf564ddc93c20f4509b89581: Deleting consensus metadata
I20260806 22:45:01.016985 25214 ts_tablet_manager.cc:1905] T bfee928f7afa468c822d99451379d867 P 0a55364713bd4ae9b31710ad0326e296: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:45:01.017426 24476 catalog_manager.cc:4841] TS fd13146fdf564ddc93c20f4509b89581 (127.18.45.194:46189): tablet bfee928f7afa468c822d99451379d867 (table test-table [id=6bf72dc0eec2411b8aa2947eaa7ee383]) successfully deleted
I20260806 22:45:01.019663 25214 ts_tablet_manager.cc:1918] T bfee928f7afa468c822d99451379d867 P 0a55364713bd4ae9b31710ad0326e296: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:45:01.019781 25214 log.cc:1196] T bfee928f7afa468c822d99451379d867 P 0a55364713bd4ae9b31710ad0326e296: Deleting WAL directory at /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786056275803658-18615-0/minicluster-data/ts-2/wal/wals/bfee928f7afa468c822d99451379d867
I20260806 22:45:01.020169 25214 ts_tablet_manager.cc:1939] T bfee928f7afa468c822d99451379d867 P 0a55364713bd4ae9b31710ad0326e296: Deleting consensus metadata
I20260806 22:45:01.020895 24476 catalog_manager.cc:4841] TS 0a55364713bd4ae9b31710ad0326e296 (127.18.45.195:43841): tablet bfee928f7afa468c822d99451379d867 (table test-table [id=6bf72dc0eec2411b8aa2947eaa7ee383]) successfully deleted
W20260806 22:45:01.021065 24477 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.45.193:37491 (error 108)
I20260806 22:45:01.021831 25175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:01.021650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.193:37491 (local address 127.0.0.1:56076)
0806 22:45:01.021723 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:01.021774 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.193:37491: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260806 22:45:01.021926 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 24775
W20260806 22:45:01.021971 24477 catalog_manager.cc:4584] TS e8d589e6c8a6423da76d4b3080cb651a (127.18.45.193:37491): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bfee928f7afa468c822d99451379d867: Network error: Client connection negotiation failed: client connection to 127.18.45.193:37491: connect: Connection refused (error 111)
I20260806 22:45:01.038832 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 24982
I20260806 22:45:01.049463 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 24452
2026-08-06T22:45:01Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2391 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-06T22:45:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:45:01Z Disabled control of system clock
I20260806 22:45:01.094067 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056275803658-18615-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056275803658-18615-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056275803658-18615-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:43665
--webserver_interface=127.18.45.254
--webserver_port=0
--builtin_ntp_servers=127.18.45.212:43181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.45.254:43665
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 22:45:01.192370 25225 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:45:01.192668 25225 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:45:01.192695 25225 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:45:01.194069 25225 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:45:01.194124 25225 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:45:01.194137 25225 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:45:01.194150 25225 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:45:01.194164 25225 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:45:01.195786 25225 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:43181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056275803658-18615-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056275803658-18615-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.45.254:43665
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:43665
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.45.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25225
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056275803658-18615-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:45:01.196148 25225 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:45:01.196578 25225 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 22:45:01.202603 25225 server_base.cc:1034] running on GCE node
W20260806 22:45:01.202577 25237 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:01.202575 25232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:01.202567 25233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:01.203214 25225 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:45:01.203653 25225 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:45:01.204855 25225 hybrid_clock.cc:648] HybridClock initialized: now 1786056301204839 us; error 53 us; skew 500 ppm
I20260806 22:45:01.206724 25225 webserver.cc:460] Webserver started at http://127.18.45.254:36341/ using document root <none> and password file <none>
I20260806 22:45:01.207139 25225 fs_manager.cc:359] Metadata directory not provided
I20260806 22:45:01.207226 25225 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:45:01.207415 25225 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:45:01.208753 25225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056275803658-18615-0/minicluster-data/master-0/data/instance:
uuid: "78b5535782b9415db30a074b24b3383f"
format_stamp: "Formatted at 2026-08-06 22:45:01 on dist-test-slave-h3ft"
server_key: "a3ed12c933dffe6d12ca01025d0618b5"
server_key_iv: "e5075ab1f4cee3cc9670623e20faa5e4"
server_key_version: "encryptionkey@0"
I20260806 22:45:01.209182 25225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056275803658-18615-0/minicluster-data/master-0/wal/instance:
uuid: "78b5535782b9415db30a074b24b3383f"
format_stamp: "Formatted at 2026-08-06 22:45:01 on dist-test-slave-h3ft"
server_key: "a3ed12c933dffe6d12ca01025d0618b5"
server_key_iv: "e5075ab1f4cee3cc9670623e20faa5e4"
server_key_version: "encryptionkey@0"
I20260806 22:45:01.210734 25225 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:45:01.211666 25243 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:01.211863 25225 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:45:01.211983 25225 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056275803658-18615-0/minicluster-data/master-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056275803658-18615-0/minicluster-data/master-0/wal
uuid: "78b5535782b9415db30a074b24b3383f"
format_stamp: "Formatted at 2026-08-06 22:45:01 on dist-test-slave-h3ft"
server_key: "a3ed12c933dffe6d12ca01025d0618b5"
server_key_iv: "e5075ab1f4cee3cc9670623e20faa5e4"
server_key_version: "encryptionkey@0"
I20260806 22:45:01.212070 25225 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056275803658-18615-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056275803658-18615-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056275803658-18615-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:45:01.216059 25225 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:45:01.216420 25225 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:45:01.216593 25225 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:45:01.221812 25225 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.254:43665
I20260806 22:45:01.221860 25327 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.254:43665 every 8 connection(s)
I20260806 22:45:01.222325 25225 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
I20260806 22:45:01.223342 25328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:45:01.227110 25328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78b5535782b9415db30a074b24b3383f: Bootstrap starting.
I20260806 22:45:01.227945 25328 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 78b5535782b9415db30a074b24b3383f: Neither blocks nor log segments found. Creating new log.
I20260806 22:45:01.228327 25328 log.cc:824] T 00000000000000000000000000000000 P 78b5535782b9415db30a074b24b3383f: Log is configured to *not* fsync() on all Append() calls
I20260806 22:45:01.229218 25328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78b5535782b9415db30a074b24b3383f: No bootstrap required, opened a new log
I20260806 22:45:01.230964 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 25225
I20260806 22:45:01.231163 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056275803658-18615-0/minicluster-data/master-0/wal/instance
I20260806 22:45:01.231385 18615 external_mini_cluster.cc:1428] Setting key 89c738e319f5d44738e02b28772c329f
I20260806 22:45:01.231467 25328 raft_consensus.cc:348] T 00000000000000000000000000000000 P 78b5535782b9415db30a074b24b3383f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78b5535782b9415db30a074b24b3383f" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 43665 } }
I20260806 22:45:01.231725 25328 raft_consensus.cc:374] T 00000000000000000000000000000000 P 78b5535782b9415db30a074b24b3383f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:45:01.231775 25328 raft_consensus.cc:729] T 00000000000000000000000000000000 P 78b5535782b9415db30a074b24b3383f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78b5535782b9415db30a074b24b3383f, State: Initialized, Role: FOLLOWER
I20260806 22:45:01.231999 25328 consensus_queue.cc:260] T 00000000000000000000000000000000 P 78b5535782b9415db30a074b24b3383f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78b5535782b9415db30a074b24b3383f" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 43665 } }
I20260806 22:45:01.232132 25328 raft_consensus.cc:388] T 00000000000000000000000000000000 P 78b5535782b9415db30a074b24b3383f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:45:01.232178 25328 raft_consensus.cc:482] T 00000000000000000000000000000000 P 78b5535782b9415db30a074b24b3383f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:45:01.232228 25328 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 78b5535782b9415db30a074b24b3383f [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:45:01.233225 25328 raft_consensus.cc:504] T 00000000000000000000000000000000 P 78b5535782b9415db30a074b24b3383f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78b5535782b9415db30a074b24b3383f" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 43665 } }
I20260806 22:45:01.233368 25328 leader_election.cc:302] T 00000000000000000000000000000000 P 78b5535782b9415db30a074b24b3383f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 78b5535782b9415db30a074b24b3383f; no voters:
I20260806 22:45:01.233573 25328 leader_election.cc:288] T 00000000000000000000000000000000 P 78b5535782b9415db30a074b24b3383f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:45:01.233740 25335 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 78b5535782b9415db30a074b24b3383f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:45:01.233932 25335 raft_consensus.cc:686] T 00000000000000000000000000000000 P 78b5535782b9415db30a074b24b3383f [term 1 LEADER]: Becoming Leader. State: Replica: 78b5535782b9415db30a074b24b3383f, State: Running, Role: LEADER
I20260806 22:45:01.233974 25328 sys_catalog.cc:564] T 00000000000000000000000000000000 P 78b5535782b9415db30a074b24b3383f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:45:01.234045 25335 consensus_queue.cc:237] T 00000000000000000000000000000000 P 78b5535782b9415db30a074b24b3383f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78b5535782b9415db30a074b24b3383f" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 43665 } }
I20260806 22:45:01.234825 25333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:01.232120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47806 (local address 127.18.45.254:43665)
0806 22:45:01.232472 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:01.232484 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:01.232527 (+ 43us) server_negotiation.cc:407] Connection header received
0806 22:45:01.232696 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:01.232706 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:01.232764 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:01.232962 (+ 198us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:01.234005 (+ 1043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:01.234010 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:01.234011 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:45:01.234017 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:01.234081 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:45:01.234099 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:01.234149 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:01.234423 (+ 274us) server_negotiation.cc:299] Negotiation successful
0806 22:45:01.234515 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":153,"threads_started":1}
I20260806 22:45:01.235944 25336 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78b5535782b9415db30a074b24b3383f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "78b5535782b9415db30a074b24b3383f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78b5535782b9415db30a074b24b3383f" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 43665 } } }
I20260806 22:45:01.236060 25336 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78b5535782b9415db30a074b24b3383f [sys.catalog]: This master's current role is: LEADER
I20260806 22:45:01.236336 25337 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78b5535782b9415db30a074b24b3383f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 78b5535782b9415db30a074b24b3383f. Latest consensus state: current_term: 1 leader_uuid: "78b5535782b9415db30a074b24b3383f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78b5535782b9415db30a074b24b3383f" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 43665 } } }
I20260806 22:45:01.236481 25337 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78b5535782b9415db30a074b24b3383f [sys.catalog]: This master's current role is: LEADER
W20260806 22:45:01.236658 25357 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 78b5535782b9415db30a074b24b3383f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:45:01.236732 25357 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:45:01.236799 25359 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:45:01.236990 25359 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:45:01.238723 25359 catalog_manager.cc:1347] Generated new cluster ID: 3164da93e96a4236bf6979766fa2b4c9
I20260806 22:45:01.238778 25359 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:45:01.259860 25359 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:45:01.260179 25359 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:45:01.264860 25359 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 78b5535782b9415db30a074b24b3383f: Generated new TSK 0
I20260806 22:45:01.265117 25359 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:45:01.270627 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056275803658-18615-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056275803658-18615-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056275803658-18615-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.193:0
--local_ip_for_outbound_sockets=127.18.45.193
--webserver_interface=127.18.45.193
--webserver_port=0
--tserver_master_addrs=127.18.45.254:43665
--builtin_ntp_servers=127.18.45.212:43181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 22:45:01.400384 25362 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:45:01.400660 25362 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:45:01.400687 25362 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:45:01.402104 25362 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:45:01.402155 25362 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:45:01.402185 25362 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.193
I20260806 22:45:01.404379 25362 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:43181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056275803658-18615-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056275803658-18615-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.45.193
--webserver_port=0
--tserver_master_addrs=127.18.45.254:43665
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25362
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.193
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056275803658-18615-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:45:01.404783 25362 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:45:01.405301 25362 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:45:01.413146 25368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:01.413094 25367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:01.413513 25371 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:01.413870 25362 server_base.cc:1034] running on GCE node
I20260806 22:45:01.414431 25362 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:45:01.414829 25362 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:45:01.416046 25362 hybrid_clock.cc:648] HybridClock initialized: now 1786056301416021 us; error 47 us; skew 500 ppm
I20260806 22:45:01.417948 25362 webserver.cc:460] Webserver started at http://127.18.45.193:45743/ using document root <none> and password file <none>
I20260806 22:45:01.418311 25362 fs_manager.cc:359] Metadata directory not provided
I20260806 22:45:01.418368 25362 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:45:01.418560 25362 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:45:01.420099 25362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056275803658-18615-0/minicluster-data/ts-0/data/instance:
uuid: "58763309251d428fa7ae4a1d2eb380bd"
format_stamp: "Formatted at 2026-08-06 22:45:01 on dist-test-slave-h3ft"
server_key: "614d87b72ef2f09710e632b442893453"
server_key_iv: "98a3205996862a4f00c7982f2748b805"
server_key_version: "encryptionkey@0"
I20260806 22:45:01.420560 25362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056275803658-18615-0/minicluster-data/ts-0/wal/instance:
uuid: "58763309251d428fa7ae4a1d2eb380bd"
format_stamp: "Formatted at 2026-08-06 22:45:01 on dist-test-slave-h3ft"
server_key: "614d87b72ef2f09710e632b442893453"
server_key_iv: "98a3205996862a4f00c7982f2748b805"
server_key_version: "encryptionkey@0"
I20260806 22:45:01.422297 25362 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:45:01.423290 25380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:01.423470 25362 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:45:01.423602 25362 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056275803658-18615-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056275803658-18615-0/minicluster-data/ts-0/wal
uuid: "58763309251d428fa7ae4a1d2eb380bd"
format_stamp: "Formatted at 2026-08-06 22:45:01 on dist-test-slave-h3ft"
server_key: "614d87b72ef2f09710e632b442893453"
server_key_iv: "98a3205996862a4f00c7982f2748b805"
server_key_version: "encryptionkey@0"
I20260806 22:45:01.423692 25362 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056275803658-18615-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056275803658-18615-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056275803658-18615-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:45:01.427836 25362 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:45:01.428241 25362 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:45:01.428407 25362 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:45:01.428689 25362 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:45:01.429116 25362 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:45:01.429177 25362 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:01.429224 25362 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:45:01.429250 25362 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:01.438216 25362 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.193:45967
I20260806 22:45:01.438261 25554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.193:45967 every 8 connection(s)
I20260806 22:45:01.438771 25362 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
I20260806 22:45:01.440028 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 25362
I20260806 22:45:01.440161 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056275803658-18615-0/minicluster-data/ts-0/wal/instance
I20260806 22:45:01.440409 18615 external_mini_cluster.cc:1428] Setting key 4b67ad9d04d8dabd3acc189e68a31e79
I20260806 22:45:01.441738 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056275803658-18615-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056275803658-18615-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056275803658-18615-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.194:0
--local_ip_for_outbound_sockets=127.18.45.194
--webserver_interface=127.18.45.194
--webserver_port=0
--tserver_master_addrs=127.18.45.254:43665
--builtin_ntp_servers=127.18.45.212:43181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260806 22:45:01.443567 25333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:01.439852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:57665 (local address 127.18.45.254:43665)
0806 22:45:01.439976 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:01.439979 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:01.440341 (+ 362us) server_negotiation.cc:407] Connection header received
0806 22:45:01.441192 (+ 851us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:01.441193 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:01.441222 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:01.441290 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:01.443175 (+ 1885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:01.443177 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:01.443178 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:45:01.443183 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:01.443206 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:45:01.443212 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:01.443299 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:01.443440 (+ 141us) server_negotiation.cc:299] Negotiation successful
0806 22:45:01.443468 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 22:45:01.443831 25559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:01.439816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:43665 (local address 127.18.45.193:57665)
0806 22:45:01.440090 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:01.440179 (+ 89us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:01.441063 (+ 884us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:01.441528 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:01.441532 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:01.441544 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:01.442852 (+ 1308us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:45:01.442853 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:01.442895 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:45:01.442960 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:45:01.443297 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:45:01.443307 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:45:01.443312 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:45:01.443431 (+ 119us) client_negotiation.cc:240] Negotiation successful
0806 22:45:01.443525 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":115,"threads_started":1}
I20260806 22:45:01.444548 25555 heartbeater.cc:347] Connected to a master server at 127.18.45.254:43665
I20260806 22:45:01.444687 25555 heartbeater.cc:464] Registering TS with master...
I20260806 22:45:01.444895 25555 heartbeater.cc:511] Master 127.18.45.254:43665 requested a full tablet report, sending...
I20260806 22:45:01.445591 25272 ts_manager.cc:194] Registered new tserver with Master: 58763309251d428fa7ae4a1d2eb380bd (127.18.45.193:45967)
W20260806 22:45:01.544265 25561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:45:01.544591 25561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:45:01.544636 25561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:45:01.546046 25561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:45:01.546128 25561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:45:01.546176 25561 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.194
I20260806 22:45:01.547809 25561 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:43181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056275803658-18615-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056275803658-18615-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.45.194
--webserver_port=0
--tserver_master_addrs=127.18.45.254:43665
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25561
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.194
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056275803658-18615-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:45:01.548168 25561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:45:01.548578 25561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:45:01.555337 25568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:01.555364 25569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:01.555337 25572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:01.555687 25561 server_base.cc:1034] running on GCE node
I20260806 22:45:01.555976 25561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:45:01.556313 25561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:45:01.557528 25561 hybrid_clock.cc:648] HybridClock initialized: now 1786056301557515 us; error 54 us; skew 500 ppm
I20260806 22:45:01.559422 25561 webserver.cc:460] Webserver started at http://127.18.45.194:33485/ using document root <none> and password file <none>
I20260806 22:45:01.559769 25561 fs_manager.cc:359] Metadata directory not provided
I20260806 22:45:01.559898 25561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:45:01.560124 25561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:45:01.561532 25561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056275803658-18615-0/minicluster-data/ts-1/data/instance:
uuid: "4c50083a1b3a4f56955c8aa9d87d23ad"
format_stamp: "Formatted at 2026-08-06 22:45:01 on dist-test-slave-h3ft"
server_key: "807636c5b645c0d74ef4a6ba6d8f1503"
server_key_iv: "0ef6d2d147f3c1032462216651373cde"
server_key_version: "encryptionkey@0"
I20260806 22:45:01.562011 25561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056275803658-18615-0/minicluster-data/ts-1/wal/instance:
uuid: "4c50083a1b3a4f56955c8aa9d87d23ad"
format_stamp: "Formatted at 2026-08-06 22:45:01 on dist-test-slave-h3ft"
server_key: "807636c5b645c0d74ef4a6ba6d8f1503"
server_key_iv: "0ef6d2d147f3c1032462216651373cde"
server_key_version: "encryptionkey@0"
I20260806 22:45:01.563994 25561 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260806 22:45:01.565193 25579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:01.565482 25561 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:45:01.565629 25561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056275803658-18615-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056275803658-18615-0/minicluster-data/ts-1/wal
uuid: "4c50083a1b3a4f56955c8aa9d87d23ad"
format_stamp: "Formatted at 2026-08-06 22:45:01 on dist-test-slave-h3ft"
server_key: "807636c5b645c0d74ef4a6ba6d8f1503"
server_key_iv: "0ef6d2d147f3c1032462216651373cde"
server_key_version: "encryptionkey@0"
I20260806 22:45:01.565727 25561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056275803658-18615-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056275803658-18615-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056275803658-18615-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:45:01.569944 25561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:45:01.570317 25561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:45:01.570482 25561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:45:01.570741 25561 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:45:01.571185 25561 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:45:01.571244 25561 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:01.571290 25561 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:45:01.571316 25561 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:01.578927 25561 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.194:33963
I20260806 22:45:01.578991 25740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.194:33963 every 8 connection(s)
I20260806 22:45:01.579459 25561 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
I20260806 22:45:01.580654 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 25561
I20260806 22:45:01.580801 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056275803658-18615-0/minicluster-data/ts-1/wal/instance
I20260806 22:45:01.581072 18615 external_mini_cluster.cc:1428] Setting key aa5c1cef9c6feafd64de8c9047a53f29
I20260806 22:45:01.582300 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056275803658-18615-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056275803658-18615-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056275803658-18615-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.195:0
--local_ip_for_outbound_sockets=127.18.45.195
--webserver_interface=127.18.45.195
--webserver_port=0
--tserver_master_addrs=127.18.45.254:43665
--builtin_ntp_servers=127.18.45.212:43181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260806 22:45:01.583803 25333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:01.580520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:43719 (local address 127.18.45.254:43665)
0806 22:45:01.580608 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:01.580612 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:01.580939 (+ 327us) server_negotiation.cc:407] Connection header received
0806 22:45:01.581748 (+ 809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:01.581750 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:01.581759 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:01.581828 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:01.583461 (+ 1633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:01.583463 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:01.583464 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:45:01.583470 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:01.583495 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:45:01.583500 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:01.583578 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:01.583694 (+ 116us) server_negotiation.cc:299] Negotiation successful
0806 22:45:01.583714 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 22:45:01.584225 25743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:01.580448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:43665 (local address 127.18.45.194:43719)
0806 22:45:01.580747 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:01.580806 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:01.581627 (+ 821us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:01.581918 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:01.581923 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:01.581936 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:01.583246 (+ 1310us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:45:01.583247 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:01.583284 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:45:01.583320 (+ 36us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:45:01.583590 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:45:01.583598 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:45:01.583602 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:45:01.583709 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 22:45:01.583795 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":139,"threads_started":1}
I20260806 22:45:01.585047 25741 heartbeater.cc:347] Connected to a master server at 127.18.45.254:43665
I20260806 22:45:01.585180 25741 heartbeater.cc:464] Registering TS with master...
I20260806 22:45:01.585394 25741 heartbeater.cc:511] Master 127.18.45.254:43665 requested a full tablet report, sending...
I20260806 22:45:01.585870 25272 ts_manager.cc:194] Registered new tserver with Master: 4c50083a1b3a4f56955c8aa9d87d23ad (127.18.45.194:33963)
W20260806 22:45:01.686013 25745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:45:01.686594 25745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:45:01.686651 25745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:45:01.688702 25745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:45:01.688781 25745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:45:01.688820 25745 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.195
I20260806 22:45:01.691108 25745 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:43181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056275803658-18615-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056275803658-18615-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.45.195
--webserver_port=0
--tserver_master_addrs=127.18.45.254:43665
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25745
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.195
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056275803658-18615-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:45:01.691509 25745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:45:01.691941 25745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:45:01.698580 25750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:01.698633 25751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:01.698776 25745 server_base.cc:1034] running on GCE node
W20260806 22:45:01.698577 25754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:01.699275 25745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:45:01.699604 25745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:45:01.700788 25745 hybrid_clock.cc:648] HybridClock initialized: now 1786056301700738 us; error 72 us; skew 500 ppm
I20260806 22:45:01.702737 25745 webserver.cc:460] Webserver started at http://127.18.45.195:40247/ using document root <none> and password file <none>
I20260806 22:45:01.703157 25745 fs_manager.cc:359] Metadata directory not provided
I20260806 22:45:01.703222 25745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:45:01.703400 25745 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:45:01.704793 25745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056275803658-18615-0/minicluster-data/ts-2/data/instance:
uuid: "6cb75155690f4d2eb5c7d3bdc089585e"
format_stamp: "Formatted at 2026-08-06 22:45:01 on dist-test-slave-h3ft"
server_key: "5ceec6cf2736c0768ee000e33c4076c5"
server_key_iv: "811e09b097651dbdcb8c88fbde6fda8f"
server_key_version: "encryptionkey@0"
I20260806 22:45:01.705251 25745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056275803658-18615-0/minicluster-data/ts-2/wal/instance:
uuid: "6cb75155690f4d2eb5c7d3bdc089585e"
format_stamp: "Formatted at 2026-08-06 22:45:01 on dist-test-slave-h3ft"
server_key: "5ceec6cf2736c0768ee000e33c4076c5"
server_key_iv: "811e09b097651dbdcb8c88fbde6fda8f"
server_key_version: "encryptionkey@0"
I20260806 22:45:01.707161 25745 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260806 22:45:01.708449 25765 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:01.708722 25745 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:45:01.708858 25745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056275803658-18615-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056275803658-18615-0/minicluster-data/ts-2/wal
uuid: "6cb75155690f4d2eb5c7d3bdc089585e"
format_stamp: "Formatted at 2026-08-06 22:45:01 on dist-test-slave-h3ft"
server_key: "5ceec6cf2736c0768ee000e33c4076c5"
server_key_iv: "811e09b097651dbdcb8c88fbde6fda8f"
server_key_version: "encryptionkey@0"
I20260806 22:45:01.708961 25745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056275803658-18615-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056275803658-18615-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056275803658-18615-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:45:01.713397 25745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:45:01.713807 25745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:45:01.713989 25745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:45:01.714275 25745 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:45:01.714751 25745 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:45:01.714811 25745 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:01.714859 25745 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:45:01.714902 25745 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:01.723680 25745 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.195:45801
I20260806 22:45:01.723896 25923 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.195:45801 every 8 connection(s)
I20260806 22:45:01.724273 25745 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
I20260806 22:45:01.728672 25333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:01.725374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.195:35901 (local address 127.18.45.254:43665)
0806 22:45:01.725537 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:01.725540 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:01.725758 (+ 218us) server_negotiation.cc:407] Connection header received
0806 22:45:01.726553 (+ 795us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:01.726555 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:01.726565 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:01.726617 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:01.728292 (+ 1675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:01.728295 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:01.728296 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:45:01.728302 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:01.728330 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:45:01.728335 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:01.728417 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:01.728547 (+ 130us) server_negotiation.cc:299] Negotiation successful
0806 22:45:01.728571 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260806 22:45:01.728963 25928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:01.725241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:43665 (local address 127.18.45.195:35901)
0806 22:45:01.725589 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:01.725646 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:01.726404 (+ 758us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:01.726725 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:01.726731 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:01.726750 (+ 19us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:01.728072 (+ 1322us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:45:01.728073 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:01.728114 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:45:01.728156 (+ 42us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:45:01.728413 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:45:01.728422 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:45:01.728427 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:45:01.728555 (+ 128us) client_negotiation.cc:240] Negotiation successful
0806 22:45:01.728643 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":141,"threads_started":1}
I20260806 22:45:01.729583 25925 heartbeater.cc:347] Connected to a master server at 127.18.45.254:43665
I20260806 22:45:01.729712 25925 heartbeater.cc:464] Registering TS with master...
I20260806 22:45:01.729874 25925 heartbeater.cc:511] Master 127.18.45.254:43665 requested a full tablet report, sending...
I20260806 22:45:01.730405 25272 ts_manager.cc:194] Registered new tserver with Master: 6cb75155690f4d2eb5c7d3bdc089585e (127.18.45.195:45801)
I20260806 22:45:01.730950 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 25745
I20260806 22:45:01.731130 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056275803658-18615-0/minicluster-data/ts-2/wal/instance
I20260806 22:45:01.731405 18615 external_mini_cluster.cc:1428] Setting key 76c4ece50d1cea5ca4ca2ac9166a5cef
I20260806 22:45:01.733001 18615 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:45:01.736549 25333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:01.734915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47818 (local address 127.18.45.254:43665)
0806 22:45:01.734993 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:01.735031 (+ 38us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:01.735207 (+ 176us) server_negotiation.cc:407] Connection header received
0806 22:45:01.735329 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:01.735329 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:01.735339 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:01.735411 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:01.736168 (+ 757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:01.736169 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:01.736170 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:45:01.736177 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:01.736204 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:45:01.736211 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:01.736282 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:01.736424 (+ 142us) server_negotiation.cc:299] Negotiation successful
0806 22:45:01.736450 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 22:45:01.738719 25272 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47818:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:45:01.740991 25272 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:45:01.746757 25935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:01.744050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.195:45801 (local address 127.0.0.1:48768)
0806 22:45:01.744749 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:01.744758 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:01.744806 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:01.745256 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:01.745263 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:01.745271 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:01.746078 (+ 807us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:45:01.746078 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:01.746134 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:45:01.746170 (+ 36us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:45:01.746490 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:45:01.746498 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:45:01.746503 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:45:01.746587 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 22:45:01.746609 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":688,"mutex_wait_us":2,"thread_start_us":83,"threads_started":1}
I20260806 22:45:01.746941 25940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:01.744291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42656 (local address 127.18.45.194:33963)
0806 22:45:01.744704 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:01.744709 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:01.744722 (+ 13us) server_negotiation.cc:407] Connection header received
0806 22:45:01.744762 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:01.744765 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:01.744783 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:01.744871 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:01.746346 (+ 1475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:01.746355 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:01.746356 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:45:01.746362 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:01.746408 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:45:01.746432 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:01.746524 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:01.746795 (+ 271us) server_negotiation.cc:299] Negotiation successful
0806 22:45:01.746819 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":129,"threads_started":1}
I20260806 22:45:01.746757 25936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:01.744047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.194:33963 (local address 127.0.0.1:42656)
0806 22:45:01.744453 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:01.744513 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:01.744596 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:01.745697 (+ 1101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:01.745698 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:01.745701 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:01.746220 (+ 519us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:45:01.746221 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:01.746227 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:45:01.746240 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:45:01.746549 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:45:01.746550 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:45:01.746551 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 22:45:01.746597 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 22:45:01.746610 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":108,"threads_started":1}
I20260806 22:45:01.747646 25939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:01.744285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48768 (local address 127.18.45.195:45801)
0806 22:45:01.744587 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:01.744594 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:01.744817 (+ 223us) server_negotiation.cc:407] Connection header received
0806 22:45:01.745079 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:01.745095 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:01.745113 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:01.745233 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:01.746286 (+ 1053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:01.746302 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:01.746302 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:45:01.746309 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:01.746362 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:45:01.746408 (+ 46us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:01.746482 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:01.746620 (+ 138us) server_negotiation.cc:299] Negotiation successful
0806 22:45:01.746644 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":125,"threads_started":1}
I20260806 22:45:01.747774 25934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:01.744050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.193:45967 (local address 127.0.0.1:39850)
0806 22:45:01.744569 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:01.744586 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:01.744670 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:01.746620 (+ 1950us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:01.746622 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:01.746626 (+ 4us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:01.747202 (+ 576us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:45:01.747203 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:01.747214 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:45:01.747231 (+ 17us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:45:01.747600 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:45:01.747601 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:45:01.747602 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 22:45:01.747677 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 22:45:01.747695 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"spinlock_wait_cycles":896,"thread_start_us":87,"threads_started":1}
I20260806 22:45:01.748072 25941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:01.744441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39850 (local address 127.18.45.193:45967)
0806 22:45:01.746024 (+ 1583us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:01.746030 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:01.746043 (+ 13us) server_negotiation.cc:407] Connection header received
0806 22:45:01.746075 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:01.746078 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:01.746089 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:01.746180 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:01.747340 (+ 1160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:01.747354 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:01.747355 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:45:01.747361 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:01.747408 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:45:01.747448 (+ 40us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:01.747529 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:01.747898 (+ 369us) server_negotiation.cc:299] Negotiation successful
0806 22:45:01.747920 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1510,"thread_start_us":99,"threads_started":1}
I20260806 22:45:01.748245 25645 tablet_service.cc:1467] Processing CreateTablet for tablet 334570b879a549b581ac136abbfb43cd (DEFAULT_TABLE table=test-table [id=c0e07e5e63304bd3b7d7587827cfb8e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:45:01.748450 25825 tablet_service.cc:1467] Processing CreateTablet for tablet 334570b879a549b581ac136abbfb43cd (DEFAULT_TABLE table=test-table [id=c0e07e5e63304bd3b7d7587827cfb8e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:45:01.748818 25645 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 334570b879a549b581ac136abbfb43cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:45:01.748865 25825 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 334570b879a549b581ac136abbfb43cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:45:01.748890 25454 tablet_service.cc:1467] Processing CreateTablet for tablet 334570b879a549b581ac136abbfb43cd (DEFAULT_TABLE table=test-table [id=c0e07e5e63304bd3b7d7587827cfb8e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:45:01.749318 25454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 334570b879a549b581ac136abbfb43cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:45:01.752550 25942 tablet_bootstrap.cc:492] T 334570b879a549b581ac136abbfb43cd P 6cb75155690f4d2eb5c7d3bdc089585e: Bootstrap starting.
I20260806 22:45:01.753319 25943 tablet_bootstrap.cc:492] T 334570b879a549b581ac136abbfb43cd P 4c50083a1b3a4f56955c8aa9d87d23ad: Bootstrap starting.
I20260806 22:45:01.754127 25942 tablet_bootstrap.cc:654] T 334570b879a549b581ac136abbfb43cd P 6cb75155690f4d2eb5c7d3bdc089585e: Neither blocks nor log segments found. Creating new log.
I20260806 22:45:01.754150 25943 tablet_bootstrap.cc:654] T 334570b879a549b581ac136abbfb43cd P 4c50083a1b3a4f56955c8aa9d87d23ad: Neither blocks nor log segments found. Creating new log.
I20260806 22:45:01.754585 25943 log.cc:824] T 334570b879a549b581ac136abbfb43cd P 4c50083a1b3a4f56955c8aa9d87d23ad: Log is configured to *not* fsync() on all Append() calls
I20260806 22:45:01.754585 25942 log.cc:824] T 334570b879a549b581ac136abbfb43cd P 6cb75155690f4d2eb5c7d3bdc089585e: Log is configured to *not* fsync() on all Append() calls
I20260806 22:45:01.755120 25945 tablet_bootstrap.cc:492] T 334570b879a549b581ac136abbfb43cd P 58763309251d428fa7ae4a1d2eb380bd: Bootstrap starting.
I20260806 22:45:01.755604 25943 tablet_bootstrap.cc:492] T 334570b879a549b581ac136abbfb43cd P 4c50083a1b3a4f56955c8aa9d87d23ad: No bootstrap required, opened a new log
I20260806 22:45:01.755703 25943 ts_tablet_manager.cc:1397] T 334570b879a549b581ac136abbfb43cd P 4c50083a1b3a4f56955c8aa9d87d23ad: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:45:01.755766 25942 tablet_bootstrap.cc:492] T 334570b879a549b581ac136abbfb43cd P 6cb75155690f4d2eb5c7d3bdc089585e: No bootstrap required, opened a new log
I20260806 22:45:01.755846 25942 ts_tablet_manager.cc:1397] T 334570b879a549b581ac136abbfb43cd P 6cb75155690f4d2eb5c7d3bdc089585e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:45:01.755965 25945 tablet_bootstrap.cc:654] T 334570b879a549b581ac136abbfb43cd P 58763309251d428fa7ae4a1d2eb380bd: Neither blocks nor log segments found. Creating new log.
I20260806 22:45:01.756368 25945 log.cc:824] T 334570b879a549b581ac136abbfb43cd P 58763309251d428fa7ae4a1d2eb380bd: Log is configured to *not* fsync() on all Append() calls
I20260806 22:45:01.757679 25945 tablet_bootstrap.cc:492] T 334570b879a549b581ac136abbfb43cd P 58763309251d428fa7ae4a1d2eb380bd: No bootstrap required, opened a new log
I20260806 22:45:01.757768 25945 ts_tablet_manager.cc:1397] T 334570b879a549b581ac136abbfb43cd P 58763309251d428fa7ae4a1d2eb380bd: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:45:01.758453 25943 raft_consensus.cc:348] T 334570b879a549b581ac136abbfb43cd P 4c50083a1b3a4f56955c8aa9d87d23ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cb75155690f4d2eb5c7d3bdc089585e" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 45801 } } peers { permanent_uuid: "58763309251d428fa7ae4a1d2eb380bd" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 45967 } } peers { permanent_uuid: "4c50083a1b3a4f56955c8aa9d87d23ad" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33963 } }
I20260806 22:45:01.758666 25943 raft_consensus.cc:374] T 334570b879a549b581ac136abbfb43cd P 4c50083a1b3a4f56955c8aa9d87d23ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:45:01.758716 25943 raft_consensus.cc:729] T 334570b879a549b581ac136abbfb43cd P 4c50083a1b3a4f56955c8aa9d87d23ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c50083a1b3a4f56955c8aa9d87d23ad, State: Initialized, Role: FOLLOWER
I20260806 22:45:01.758898 25943 consensus_queue.cc:260] T 334570b879a549b581ac136abbfb43cd P 4c50083a1b3a4f56955c8aa9d87d23ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cb75155690f4d2eb5c7d3bdc089585e" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 45801 } } peers { permanent_uuid: "58763309251d428fa7ae4a1d2eb380bd" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 45967 } } peers { permanent_uuid: "4c50083a1b3a4f56955c8aa9d87d23ad" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33963 } }
I20260806 22:45:01.759300 25741 heartbeater.cc:503] Master 127.18.45.254:43665 was elected leader, sending a full tablet report...
I20260806 22:45:01.759212 25942 raft_consensus.cc:348] T 334570b879a549b581ac136abbfb43cd P 6cb75155690f4d2eb5c7d3bdc089585e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cb75155690f4d2eb5c7d3bdc089585e" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 45801 } } peers { permanent_uuid: "58763309251d428fa7ae4a1d2eb380bd" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 45967 } } peers { permanent_uuid: "4c50083a1b3a4f56955c8aa9d87d23ad" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33963 } }
I20260806 22:45:01.759387 25942 raft_consensus.cc:374] T 334570b879a549b581ac136abbfb43cd P 6cb75155690f4d2eb5c7d3bdc089585e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:45:01.759428 25942 raft_consensus.cc:729] T 334570b879a549b581ac136abbfb43cd P 6cb75155690f4d2eb5c7d3bdc089585e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cb75155690f4d2eb5c7d3bdc089585e, State: Initialized, Role: FOLLOWER
I20260806 22:45:01.759563 25942 consensus_queue.cc:260] T 334570b879a549b581ac136abbfb43cd P 6cb75155690f4d2eb5c7d3bdc089585e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cb75155690f4d2eb5c7d3bdc089585e" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 45801 } } peers { permanent_uuid: "58763309251d428fa7ae4a1d2eb380bd" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 45967 } } peers { permanent_uuid: "4c50083a1b3a4f56955c8aa9d87d23ad" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33963 } }
I20260806 22:45:01.759286 25943 ts_tablet_manager.cc:1428] T 334570b879a549b581ac136abbfb43cd P 4c50083a1b3a4f56955c8aa9d87d23ad: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260806 22:45:01.759892 25942 ts_tablet_manager.cc:1428] T 334570b879a549b581ac136abbfb43cd P 6cb75155690f4d2eb5c7d3bdc089585e: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260806 22:45:01.759927 25925 heartbeater.cc:503] Master 127.18.45.254:43665 was elected leader, sending a full tablet report...
I20260806 22:45:01.761986 25945 raft_consensus.cc:348] T 334570b879a549b581ac136abbfb43cd P 58763309251d428fa7ae4a1d2eb380bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cb75155690f4d2eb5c7d3bdc089585e" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 45801 } } peers { permanent_uuid: "58763309251d428fa7ae4a1d2eb380bd" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 45967 } } peers { permanent_uuid: "4c50083a1b3a4f56955c8aa9d87d23ad" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33963 } }
I20260806 22:45:01.762172 25945 raft_consensus.cc:374] T 334570b879a549b581ac136abbfb43cd P 58763309251d428fa7ae4a1d2eb380bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:45:01.762197 25945 raft_consensus.cc:729] T 334570b879a549b581ac136abbfb43cd P 58763309251d428fa7ae4a1d2eb380bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58763309251d428fa7ae4a1d2eb380bd, State: Initialized, Role: FOLLOWER
I20260806 22:45:01.762347 25945 consensus_queue.cc:260] T 334570b879a549b581ac136abbfb43cd P 58763309251d428fa7ae4a1d2eb380bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cb75155690f4d2eb5c7d3bdc089585e" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 45801 } } peers { permanent_uuid: "58763309251d428fa7ae4a1d2eb380bd" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 45967 } } peers { permanent_uuid: "4c50083a1b3a4f56955c8aa9d87d23ad" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33963 } }
I20260806 22:45:01.762696 25945 ts_tablet_manager.cc:1428] T 334570b879a549b581ac136abbfb43cd P 58763309251d428fa7ae4a1d2eb380bd: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260806 22:45:01.762776 25555 heartbeater.cc:503] Master 127.18.45.254:43665 was elected leader, sending a full tablet report...
W20260806 22:45:01.830190 25742 tablet.cc:2369] T 334570b879a549b581ac136abbfb43cd P 4c50083a1b3a4f56955c8aa9d87d23ad: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:45:01.925537 25955 raft_consensus.cc:482] T 334570b879a549b581ac136abbfb43cd P 6cb75155690f4d2eb5c7d3bdc089585e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:45:01.925735 25955 raft_consensus.cc:504] T 334570b879a549b581ac136abbfb43cd P 6cb75155690f4d2eb5c7d3bdc089585e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cb75155690f4d2eb5c7d3bdc089585e" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 45801 } } peers { permanent_uuid: "58763309251d428fa7ae4a1d2eb380bd" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 45967 } } peers { permanent_uuid: "4c50083a1b3a4f56955c8aa9d87d23ad" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33963 } }
I20260806 22:45:01.926326 25955 leader_election.cc:288] T 334570b879a549b581ac136abbfb43cd P 6cb75155690f4d2eb5c7d3bdc089585e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 58763309251d428fa7ae4a1d2eb380bd (127.18.45.193:45967), 4c50083a1b3a4f56955c8aa9d87d23ad (127.18.45.194:33963)
I20260806 22:45:01.928273 25928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:01.926644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.193:45967 (local address 127.18.45.195:50773)
0806 22:45:01.926751 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:01.926769 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:01.926833 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:01.927114 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:01.927116 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:01.927120 (+ 4us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:01.927726 (+ 606us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:45:01.927727 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:01.927737 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:45:01.927757 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:45:01.928095 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:45:01.928096 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:45:01.928097 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 22:45:01.928158 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 22:45:01.928176 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260806 22:45:01.928334 25941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:01.926714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.195:50773 (local address 127.18.45.193:45967)
0806 22:45:01.926823 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:01.926825 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:01.926840 (+ 15us) server_negotiation.cc:407] Connection header received
0806 22:45:01.926975 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:01.926977 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:01.926987 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:01.927103 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:01.927937 (+ 834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:01.927939 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:01.927940 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:45:01.927945 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:01.927972 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:45:01.927978 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:01.928055 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:01.928206 (+ 151us) server_negotiation.cc:299] Negotiation successful
0806 22:45:01.928226 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 22:45:01.928395 25940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:01.926805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.195:32991 (local address 127.18.45.194:33963)
0806 22:45:01.926938 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:01.926944 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:01.927097 (+ 153us) server_negotiation.cc:407] Connection header received
0806 22:45:01.927182 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:01.927183 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:01.927190 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:01.927265 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:01.928103 (+ 838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:01.928106 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:01.928106 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:45:01.928112 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:01.928134 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:45:01.928140 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:01.928200 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:01.928292 (+ 92us) server_negotiation.cc:299] Negotiation successful
0806 22:45:01.928308 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260806 22:45:01.928653 25958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:01.926649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.194:33963 (local address 127.18.45.195:32991)
0806 22:45:01.926969 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:01.926980 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:01.927103 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:01.927284 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:01.927285 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:01.927304 (+ 19us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:01.927956 (+ 652us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:45:01.927956 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:01.927964 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:45:01.927977 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:45:01.928224 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:45:01.928224 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:45:01.928225 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 22:45:01.928285 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 22:45:01.928298 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"mutex_wait_us":31,"thread_start_us":171,"threads_started":1}
I20260806 22:45:01.928977 25484 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "334570b879a549b581ac136abbfb43cd" candidate_uuid: "6cb75155690f4d2eb5c7d3bdc089585e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58763309251d428fa7ae4a1d2eb380bd" is_pre_election: true
I20260806 22:45:01.929251 25484 raft_consensus.cc:2454] T 334570b879a549b581ac136abbfb43cd P 58763309251d428fa7ae4a1d2eb380bd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6cb75155690f4d2eb5c7d3bdc089585e in term 0.
I20260806 22:45:01.929196 25675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "334570b879a549b581ac136abbfb43cd" candidate_uuid: "6cb75155690f4d2eb5c7d3bdc089585e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c50083a1b3a4f56955c8aa9d87d23ad" is_pre_election: true
I20260806 22:45:01.929369 25675 raft_consensus.cc:2454] T 334570b879a549b581ac136abbfb43cd P 4c50083a1b3a4f56955c8aa9d87d23ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6cb75155690f4d2eb5c7d3bdc089585e in term 0.
I20260806 22:45:01.929633 25769 leader_election.cc:302] T 334570b879a549b581ac136abbfb43cd P 6cb75155690f4d2eb5c7d3bdc089585e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58763309251d428fa7ae4a1d2eb380bd, 6cb75155690f4d2eb5c7d3bdc089585e; no voters:
I20260806 22:45:01.929792 25955 raft_consensus.cc:2781] T 334570b879a549b581ac136abbfb43cd P 6cb75155690f4d2eb5c7d3bdc089585e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:45:01.929857 25955 raft_consensus.cc:482] T 334570b879a549b581ac136abbfb43cd P 6cb75155690f4d2eb5c7d3bdc089585e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:45:01.929893 25955 raft_consensus.cc:3037] T 334570b879a549b581ac136abbfb43cd P 6cb75155690f4d2eb5c7d3bdc089585e [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:45:01.930882 25955 raft_consensus.cc:504] T 334570b879a549b581ac136abbfb43cd P 6cb75155690f4d2eb5c7d3bdc089585e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cb75155690f4d2eb5c7d3bdc089585e" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 45801 } } peers { permanent_uuid: "58763309251d428fa7ae4a1d2eb380bd" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 45967 } } peers { permanent_uuid: "4c50083a1b3a4f56955c8aa9d87d23ad" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33963 } }
I20260806 22:45:01.931172 25955 leader_election.cc:288] T 334570b879a549b581ac136abbfb43cd P 6cb75155690f4d2eb5c7d3bdc089585e [CANDIDATE]: Term 1 election: Requested vote from peers 58763309251d428fa7ae4a1d2eb380bd (127.18.45.193:45967), 4c50083a1b3a4f56955c8aa9d87d23ad (127.18.45.194:33963)
I20260806 22:45:01.931455 25484 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "334570b879a549b581ac136abbfb43cd" candidate_uuid: "6cb75155690f4d2eb5c7d3bdc089585e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58763309251d428fa7ae4a1d2eb380bd"
I20260806 22:45:01.931432 25675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "334570b879a549b581ac136abbfb43cd" candidate_uuid: "6cb75155690f4d2eb5c7d3bdc089585e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c50083a1b3a4f56955c8aa9d87d23ad"
I20260806 22:45:01.931577 25484 raft_consensus.cc:3037] T 334570b879a549b581ac136abbfb43cd P 58763309251d428fa7ae4a1d2eb380bd [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:45:01.931579 25675 raft_consensus.cc:3037] T 334570b879a549b581ac136abbfb43cd P 4c50083a1b3a4f56955c8aa9d87d23ad [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:45:01.932971 25484 raft_consensus.cc:2454] T 334570b879a549b581ac136abbfb43cd P 58763309251d428fa7ae4a1d2eb380bd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cb75155690f4d2eb5c7d3bdc089585e in term 1.
I20260806 22:45:01.932971 25675 raft_consensus.cc:2454] T 334570b879a549b581ac136abbfb43cd P 4c50083a1b3a4f56955c8aa9d87d23ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6cb75155690f4d2eb5c7d3bdc089585e in term 1.
I20260806 22:45:01.933308 25770 leader_election.cc:302] T 334570b879a549b581ac136abbfb43cd P 6cb75155690f4d2eb5c7d3bdc089585e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c50083a1b3a4f56955c8aa9d87d23ad, 6cb75155690f4d2eb5c7d3bdc089585e; no voters:
I20260806 22:45:01.933444 25955 raft_consensus.cc:2781] T 334570b879a549b581ac136abbfb43cd P 6cb75155690f4d2eb5c7d3bdc089585e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:45:01.933725 25955 raft_consensus.cc:686] T 334570b879a549b581ac136abbfb43cd P 6cb75155690f4d2eb5c7d3bdc089585e [term 1 LEADER]: Becoming Leader. State: Replica: 6cb75155690f4d2eb5c7d3bdc089585e, State: Running, Role: LEADER
I20260806 22:45:01.933902 25955 consensus_queue.cc:237] T 334570b879a549b581ac136abbfb43cd P 6cb75155690f4d2eb5c7d3bdc089585e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cb75155690f4d2eb5c7d3bdc089585e" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 45801 } } peers { permanent_uuid: "58763309251d428fa7ae4a1d2eb380bd" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 45967 } } peers { permanent_uuid: "4c50083a1b3a4f56955c8aa9d87d23ad" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33963 } }
I20260806 22:45:01.935221 25272 catalog_manager.cc:5495] T 334570b879a549b581ac136abbfb43cd P 6cb75155690f4d2eb5c7d3bdc089585e reported cstate change: term changed from 0 to 1, leader changed from <none> to 6cb75155690f4d2eb5c7d3bdc089585e (127.18.45.195). New cstate: current_term: 1 leader_uuid: "6cb75155690f4d2eb5c7d3bdc089585e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6cb75155690f4d2eb5c7d3bdc089585e" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 45801 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58763309251d428fa7ae4a1d2eb380bd" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 45967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c50083a1b3a4f56955c8aa9d87d23ad" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33963 } health_report { overall_health: UNKNOWN } } }
W20260806 22:45:01.939955 25558 tablet.cc:2369] T 334570b879a549b581ac136abbfb43cd P 58763309251d428fa7ae4a1d2eb380bd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:45:01.975152 25927 tablet.cc:2369] T 334570b879a549b581ac136abbfb43cd P 6cb75155690f4d2eb5c7d3bdc089585e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:45:01.978884 25939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:01.977588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48770 (local address 127.18.45.195:45801)
0806 22:45:01.977665 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:01.977668 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:01.977685 (+ 17us) server_negotiation.cc:407] Connection header received
0806 22:45:01.977734 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:01.977735 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:01.977754 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:01.977808 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:01.978593 (+ 785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:01.978594 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:01.978594 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:45:01.978598 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:01.978617 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:45:01.978621 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:01.978674 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:01.978738 (+ 64us) server_negotiation.cc:299] Negotiation successful
0806 22:45:01.978750 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 22:45:01.981504 25675 raft_consensus.cc:1261] T 334570b879a549b581ac136abbfb43cd P 4c50083a1b3a4f56955c8aa9d87d23ad [term 1 FOLLOWER]: Refusing update from remote peer 6cb75155690f4d2eb5c7d3bdc089585e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:45:01.981504 25484 raft_consensus.cc:1261] T 334570b879a549b581ac136abbfb43cd P 58763309251d428fa7ae4a1d2eb380bd [term 1 FOLLOWER]: Refusing update from remote peer 6cb75155690f4d2eb5c7d3bdc089585e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:45:01.982091 25955 consensus_queue.cc:1035] T 334570b879a549b581ac136abbfb43cd P 6cb75155690f4d2eb5c7d3bdc089585e [LEADER]: Connected to new peer: Peer: permanent_uuid: "58763309251d428fa7ae4a1d2eb380bd" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 45967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:45:01.982311 25959 consensus_queue.cc:1035] T 334570b879a549b581ac136abbfb43cd P 6cb75155690f4d2eb5c7d3bdc089585e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c50083a1b3a4f56955c8aa9d87d23ad" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:45:01.986083 25964 mvcc.cc:205] Tried to move back new op lower bound from 7315686612911972352 to 7315686612722257920. Current Snapshot: MvccSnapshot[applied={T|T < 7315686612911972352}]
I20260806 22:45:01.986294 25963 mvcc.cc:205] Tried to move back new op lower bound from 7315686612911972352 to 7315686612722257920. Current Snapshot: MvccSnapshot[applied={T|T < 7315686612911972352}]
I20260806 22:45:01.990438 25272 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47818:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:45:01.990633 25272 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47818:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:45:01.991909 25272 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 78b5535782b9415db30a074b24b3383f: Sending DeleteTablet for 3 replicas of tablet 334570b879a549b581ac136abbfb43cd
I20260806 22:45:01.992429 25825 tablet_service.cc:1514] Processing DeleteTablet for tablet 334570b879a549b581ac136abbfb43cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:45:01 UTC) from {username='slave'} at 127.0.0.1:48768
I20260806 22:45:01.992722 25645 tablet_service.cc:1514] Processing DeleteTablet for tablet 334570b879a549b581ac136abbfb43cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:45:01 UTC) from {username='slave'} at 127.0.0.1:42656
I20260806 22:45:01.992863 25454 tablet_service.cc:1514] Processing DeleteTablet for tablet 334570b879a549b581ac136abbfb43cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:45:01 UTC) from {username='slave'} at 127.0.0.1:39850
I20260806 22:45:01.993039 25973 tablet_replica.cc:330] T 334570b879a549b581ac136abbfb43cd P 6cb75155690f4d2eb5c7d3bdc089585e: stopping tablet replica
I20260806 22:45:01.993185 25974 tablet_replica.cc:330] T 334570b879a549b581ac136abbfb43cd P 4c50083a1b3a4f56955c8aa9d87d23ad: stopping tablet replica
I20260806 22:45:01.993284 25973 raft_consensus.cc:2229] T 334570b879a549b581ac136abbfb43cd P 6cb75155690f4d2eb5c7d3bdc089585e [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:45:01.993350 25974 raft_consensus.cc:2229] T 334570b879a549b581ac136abbfb43cd P 4c50083a1b3a4f56955c8aa9d87d23ad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:45:01.993528 25974 raft_consensus.cc:2258] T 334570b879a549b581ac136abbfb43cd P 4c50083a1b3a4f56955c8aa9d87d23ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:45:01.993618 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 25362
I20260806 22:45:01.993670 25973 raft_consensus.cc:2258] T 334570b879a549b581ac136abbfb43cd P 6cb75155690f4d2eb5c7d3bdc089585e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:45:02.035498 25973 ts_tablet_manager.cc:1905] T 334570b879a549b581ac136abbfb43cd P 6cb75155690f4d2eb5c7d3bdc089585e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:45:02.039212 25973 ts_tablet_manager.cc:1918] T 334570b879a549b581ac136abbfb43cd P 6cb75155690f4d2eb5c7d3bdc089585e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:45:02.039368 25973 log.cc:1196] T 334570b879a549b581ac136abbfb43cd P 6cb75155690f4d2eb5c7d3bdc089585e: Deleting WAL directory at /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056275803658-18615-0/minicluster-data/ts-2/wal/wals/334570b879a549b581ac136abbfb43cd
I20260806 22:45:02.039739 25973 ts_tablet_manager.cc:1939] T 334570b879a549b581ac136abbfb43cd P 6cb75155690f4d2eb5c7d3bdc089585e: Deleting consensus metadata
I20260806 22:45:02.040401 25245 catalog_manager.cc:4841] TS 6cb75155690f4d2eb5c7d3bdc089585e (127.18.45.195:45801): tablet 334570b879a549b581ac136abbfb43cd (table test-table [id=c0e07e5e63304bd3b7d7587827cfb8e3]) successfully deleted
I20260806 22:45:02.042291 25974 ts_tablet_manager.cc:1905] T 334570b879a549b581ac136abbfb43cd P 4c50083a1b3a4f56955c8aa9d87d23ad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:45:02.044559 25974 ts_tablet_manager.cc:1918] T 334570b879a549b581ac136abbfb43cd P 4c50083a1b3a4f56955c8aa9d87d23ad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:45:02.044685 25974 log.cc:1196] T 334570b879a549b581ac136abbfb43cd P 4c50083a1b3a4f56955c8aa9d87d23ad: Deleting WAL directory at /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786056275803658-18615-0/minicluster-data/ts-1/wal/wals/334570b879a549b581ac136abbfb43cd
I20260806 22:45:02.045029 25974 ts_tablet_manager.cc:1939] T 334570b879a549b581ac136abbfb43cd P 4c50083a1b3a4f56955c8aa9d87d23ad: Deleting consensus metadata
I20260806 22:45:02.045723 25252 catalog_manager.cc:4841] TS 4c50083a1b3a4f56955c8aa9d87d23ad (127.18.45.194:33963): tablet 334570b879a549b581ac136abbfb43cd (table test-table [id=c0e07e5e63304bd3b7d7587827cfb8e3]) successfully deleted
W20260806 22:45:02.049433 25251 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.45.193:45967 (error 108)
I20260806 22:45:02.050225 25934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:02.050019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.193:45967 (local address 127.0.0.1:39860)
0806 22:45:02.050103 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:02.050168 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.193:45967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260806 22:45:02.050307 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 25561
W20260806 22:45:02.050412 25251 catalog_manager.cc:4584] TS 58763309251d428fa7ae4a1d2eb380bd (127.18.45.193:45967): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 334570b879a549b581ac136abbfb43cd: Network error: Client connection negotiation failed: client connection to 127.18.45.193:45967: connect: Connection refused (error 111)
I20260806 22:45:02.066967 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 25745
I20260806 22:45:02.083536 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 25225
2026-08-06T22:45:02Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1039 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260806 22:45:02.122174 18615 mini_oidc.cc:152] Starting JWKS server
I20260806 22:45:02.123677 18615 webserver.cc:460] Webserver started at https://127.0.0.1:43921/ using document root <none> and password file <none>
I20260806 22:45:02.123837 18615 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 22:45:02.124125 18615 webserver.cc:460] Webserver started at http://127.0.0.1:36199/ using document root <none> and password file <none>
2026-08-06T22:45:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:45:02Z Disabled control of system clock
I20260806 22:45:02.144644 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056275803658-18615-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056275803658-18615-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056275803658-18615-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:42449
--webserver_interface=127.18.45.254
--webserver_port=0
--builtin_ntp_servers=127.18.45.212:41503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.45.254:42449
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43921/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056275803658-18615-0/testchainca.cert with env {}
W20260806 22:45:02.318161 25986 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:45:02.318643 25986 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:45:02.318698 25986 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:45:02.321235 25986 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:45:02.321358 25986 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:45:02.321386 25986 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:45:02.321412 25986 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:45:02.321448 25986 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 22:45:02.321511 25986 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 22:45:02.321601 25986 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43921/jwks.json
I20260806 22:45:02.324874 25986 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:41503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056275803658-18615-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056275803658-18615-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.45.254:42449
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:42449
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43921/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.45.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056275803658-18615-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25986
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056275803658-18615-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:45:02.325464 25986 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:45:02.325974 25986 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:45:02.336333 25991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:02.336652 25986 server_base.cc:1034] running on GCE node
W20260806 22:45:02.336333 25995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:02.336424 25992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:02.337685 25986 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:45:02.338397 25986 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:45:02.339670 25986 hybrid_clock.cc:648] HybridClock initialized: now 1786056302339576 us; error 130 us; skew 500 ppm
I20260806 22:45:02.343611 25986 webserver.cc:460] Webserver started at http://127.18.45.254:38031/ using document root <none> and password file <none>
I20260806 22:45:02.344169 25986 fs_manager.cc:359] Metadata directory not provided
I20260806 22:45:02.344280 25986 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:45:02.344590 25986 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:45:02.346926 25986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056275803658-18615-0/minicluster-data/master-0/data/instance:
uuid: "03bd3fa165254d20ba6a96aaa909fafb"
format_stamp: "Formatted at 2026-08-06 22:45:02 on dist-test-slave-h3ft"
server_key: "3252dbe6e40ee053ffb015f39e11abed"
server_key_iv: "0b86cdb04f1b1e81c23a573ef2b17af6"
server_key_version: "encryptionkey@0"
I20260806 22:45:02.347926 25986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056275803658-18615-0/minicluster-data/master-0/wal/instance:
uuid: "03bd3fa165254d20ba6a96aaa909fafb"
format_stamp: "Formatted at 2026-08-06 22:45:02 on dist-test-slave-h3ft"
server_key: "3252dbe6e40ee053ffb015f39e11abed"
server_key_iv: "0b86cdb04f1b1e81c23a573ef2b17af6"
server_key_version: "encryptionkey@0"
I20260806 22:45:02.351207 25986 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 22:45:02.352895 26004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:02.353253 25986 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:45:02.353403 25986 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056275803658-18615-0/minicluster-data/master-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056275803658-18615-0/minicluster-data/master-0/wal
uuid: "03bd3fa165254d20ba6a96aaa909fafb"
format_stamp: "Formatted at 2026-08-06 22:45:02 on dist-test-slave-h3ft"
server_key: "3252dbe6e40ee053ffb015f39e11abed"
server_key_iv: "0b86cdb04f1b1e81c23a573ef2b17af6"
server_key_version: "encryptionkey@0"
I20260806 22:45:02.353524 25986 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056275803658-18615-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056275803658-18615-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056275803658-18615-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:45:02.374126 25986 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:45:02.374785 25986 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:45:02.374992 25986 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:45:02.383620 25986 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.254:42449
I20260806 22:45:02.383729 26077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.254:42449 every 8 connection(s)
I20260806 22:45:02.384466 25986 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
I20260806 22:45:02.386839 26078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:45:02.388847 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 25986
I20260806 22:45:02.389127 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056275803658-18615-0/minicluster-data/master-0/wal/instance
I20260806 22:45:02.389495 18615 external_mini_cluster.cc:1428] Setting key 1878f1ccce24ca79d59a3fd9b43b81c7
I20260806 22:45:02.394451 26078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03bd3fa165254d20ba6a96aaa909fafb: Bootstrap starting.
I20260806 22:45:02.396278 26078 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 03bd3fa165254d20ba6a96aaa909fafb: Neither blocks nor log segments found. Creating new log.
I20260806 22:45:02.397009 26078 log.cc:824] T 00000000000000000000000000000000 P 03bd3fa165254d20ba6a96aaa909fafb: Log is configured to *not* fsync() on all Append() calls
I20260806 22:45:02.399070 26078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03bd3fa165254d20ba6a96aaa909fafb: No bootstrap required, opened a new log
I20260806 22:45:02.403263 26083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:02.390752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33624 (local address 127.18.45.254:42449)
0806 22:45:02.391257 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:02.391283 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:02.391379 (+ 96us) server_negotiation.cc:407] Connection header received
0806 22:45:02.392512 (+ 1133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:02.392538 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:02.392749 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:02.393053 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:02.393691 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:02.396260 (+ 2569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:02.398376 (+ 2116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:02.399795 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:02.400061 (+ 266us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:02.401489 (+ 1428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:02.401506 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:02.401507 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:45:02.401956 (+ 449us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:02.402075 (+ 119us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:45:02.402111 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:02.402252 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:02.402439 (+ 187us) server_negotiation.cc:299] Negotiation successful
0806 22:45:02.402666 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":254,"threads_started":1}
I20260806 22:45:02.404836 26078 raft_consensus.cc:348] T 00000000000000000000000000000000 P 03bd3fa165254d20ba6a96aaa909fafb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03bd3fa165254d20ba6a96aaa909fafb" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 42449 } }
I20260806 22:45:02.405248 26078 raft_consensus.cc:374] T 00000000000000000000000000000000 P 03bd3fa165254d20ba6a96aaa909fafb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:45:02.405324 26078 raft_consensus.cc:729] T 00000000000000000000000000000000 P 03bd3fa165254d20ba6a96aaa909fafb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03bd3fa165254d20ba6a96aaa909fafb, State: Initialized, Role: FOLLOWER
I20260806 22:45:02.405551 18615 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:45:02.405635 26078 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03bd3fa165254d20ba6a96aaa909fafb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03bd3fa165254d20ba6a96aaa909fafb" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 42449 } }
I20260806 22:45:02.405771 26078 raft_consensus.cc:388] T 00000000000000000000000000000000 P 03bd3fa165254d20ba6a96aaa909fafb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:45:02.405833 26078 raft_consensus.cc:482] T 00000000000000000000000000000000 P 03bd3fa165254d20ba6a96aaa909fafb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:45:02.405926 26078 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 03bd3fa165254d20ba6a96aaa909fafb [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:45:02.407735 26078 raft_consensus.cc:504] T 00000000000000000000000000000000 P 03bd3fa165254d20ba6a96aaa909fafb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03bd3fa165254d20ba6a96aaa909fafb" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 42449 } }
I20260806 22:45:02.407966 26078 leader_election.cc:302] T 00000000000000000000000000000000 P 03bd3fa165254d20ba6a96aaa909fafb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 03bd3fa165254d20ba6a96aaa909fafb; no voters:
I20260806 22:45:02.408317 26078 leader_election.cc:288] T 00000000000000000000000000000000 P 03bd3fa165254d20ba6a96aaa909fafb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:45:02.408572 26087 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 03bd3fa165254d20ba6a96aaa909fafb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:45:02.409150 26087 raft_consensus.cc:686] T 00000000000000000000000000000000 P 03bd3fa165254d20ba6a96aaa909fafb [term 1 LEADER]: Becoming Leader. State: Replica: 03bd3fa165254d20ba6a96aaa909fafb, State: Running, Role: LEADER
I20260806 22:45:02.409368 26087 consensus_queue.cc:237] T 00000000000000000000000000000000 P 03bd3fa165254d20ba6a96aaa909fafb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03bd3fa165254d20ba6a96aaa909fafb" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 42449 } }
I20260806 22:45:02.409405 26078 sys_catalog.cc:564] T 00000000000000000000000000000000 P 03bd3fa165254d20ba6a96aaa909fafb [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:45:02.410727 26088 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03bd3fa165254d20ba6a96aaa909fafb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "03bd3fa165254d20ba6a96aaa909fafb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03bd3fa165254d20ba6a96aaa909fafb" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 42449 } } }
I20260806 22:45:02.410912 26088 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03bd3fa165254d20ba6a96aaa909fafb [sys.catalog]: This master's current role is: LEADER
I20260806 22:45:02.411126 26089 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03bd3fa165254d20ba6a96aaa909fafb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 03bd3fa165254d20ba6a96aaa909fafb. Latest consensus state: current_term: 1 leader_uuid: "03bd3fa165254d20ba6a96aaa909fafb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03bd3fa165254d20ba6a96aaa909fafb" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 42449 } } }
I20260806 22:45:02.411303 26089 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03bd3fa165254d20ba6a96aaa909fafb [sys.catalog]: This master's current role is: LEADER
I20260806 22:45:02.411445 26095 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:45:02.412735 26095 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:45:02.422943 26095 catalog_manager.cc:1347] Generated new cluster ID: 88069554172a4bd6beb0d3729250f85f
I20260806 22:45:02.423162 26095 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:45:02.438920 26095 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:45:02.440362 26095 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:45:02.461479 26095 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 03bd3fa165254d20ba6a96aaa909fafb: Generated new TSK 0
I20260806 22:45:02.461990 26095 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 22:45:02.499655 26124 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260806 22:45:02.499972 26124 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 22:45:02.500794 26083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:02.475378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33638 (local address 127.18.45.254:42449)
0806 22:45:02.475515 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:02.475518 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:02.475889 (+ 371us) server_negotiation.cc:407] Connection header received
0806 22:45:02.476119 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:02.476121 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:02.476141 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:02.476362 (+ 221us) server_negotiation.cc:226] Negotiated authn=JWT
0806 22:45:02.476605 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:02.477838 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:02.479316 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:02.479834 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:02.479980 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:02.480558 (+ 578us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 22:45:02.500068 (+ 19510us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43921/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 22:45:02.500649 (+ 581us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33638: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43921/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":67}
W20260806 22:45:02.500941 26083 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33638: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43921/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260806 22:45:02.501662 26123 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:45:02.475260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:42449 (local address 127.0.0.1:33638)
0806 22:45:02.475756 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:02.475778 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:02.475905 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:02.476365 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:02.476366 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:02.476369 (+ 3us) client_negotiation.cc:189] Negotiated authn=JWT
0806 22:45:02.476492 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:02.476496 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:02.478040 (+ 1544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:02.478042 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:02.479152 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:02.479165 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:02.480050 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:02.480053 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:02.480164 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:02.480230 (+ 66us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 22:45:02.500484 (+ 20254us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43921/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 22:45:02.500745 (+ 261us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.18.45.254:42449: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43921/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":279,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.18.45.254:42449: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43921/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260806 22:45:02.502708 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 25986
2026-08-06T22:45:02Z chronyd exiting
I20260806 22:45:02.561954 18615 test_util.cc:175] -----------------------------------------------
I20260806 22:45:02.562144 18615 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786056275803658-18615-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (441 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260806 22:45:02.563912 18615 mini_oidc.cc:152] Starting JWKS server
I20260806 22:45:02.565184 18615 webserver.cc:460] Webserver started at https://127.0.0.1:40491/ using document root <none> and password file <none>
I20260806 22:45:02.565254 18615 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 22:45:02.565538 18615 webserver.cc:460] Webserver started at http://127.0.0.1:44329/ using document root <none> and password file <none>
2026-08-06T22:45:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:45:02Z Disabled control of system clock
I20260806 22:45:02.585762 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056275803658-18615-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056275803658-18615-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056275803658-18615-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:46119
--webserver_interface=127.18.45.254
--webserver_port=0
--builtin_ntp_servers=127.18.45.212:34089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.45.254:46119
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40491/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056275803658-18615-0/testchainca.cert with env {}
W20260806 22:45:02.762391 26133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:45:02.762921 26133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:45:02.762966 26133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:45:02.765368 26133 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:45:02.765475 26133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:45:02.765509 26133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:45:02.765527 26133 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:45:02.765540 26133 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 22:45:02.765566 26133 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 22:45:02.765591 26133 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40491/jwks.json
I20260806 22:45:02.769237 26133 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:34089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056275803658-18615-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056275803658-18615-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.45.254:46119
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:46119
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40491/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.45.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056275803658-18615-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26133
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056275803658-18615-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:45:02.769825 26133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:45:02.770316 26133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:45:02.779789 26144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:02.779776 26149 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:02.779855 26142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:02.780104 26133 server_base.cc:1034] running on GCE node
I20260806 22:45:02.780797 26133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:45:02.781467 26133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:45:02.782716 26133 hybrid_clock.cc:648] HybridClock initialized: now 1786056302782672 us; error 68 us; skew 500 ppm
I20260806 22:45:02.785512 26133 webserver.cc:460] Webserver started at http://127.18.45.254:39445/ using document root <none> and password file <none>
I20260806 22:45:02.786067 26133 fs_manager.cc:359] Metadata directory not provided
I20260806 22:45:02.786167 26133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:45:02.786458 26133 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:45:02.788645 26133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056275803658-18615-0/minicluster-data/master-0/data/instance:
uuid: "045409b588cb43ba85aae707500d8bd7"
format_stamp: "Formatted at 2026-08-06 22:45:02 on dist-test-slave-h3ft"
server_key: "869b9edd1d47da866f51686758046a91"
server_key_iv: "1c330af84a04cd528630affeb9b99920"
server_key_version: "encryptionkey@0"
I20260806 22:45:02.789390 26133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056275803658-18615-0/minicluster-data/master-0/wal/instance:
uuid: "045409b588cb43ba85aae707500d8bd7"
format_stamp: "Formatted at 2026-08-06 22:45:02 on dist-test-slave-h3ft"
server_key: "869b9edd1d47da866f51686758046a91"
server_key_iv: "1c330af84a04cd528630affeb9b99920"
server_key_version: "encryptionkey@0"
I20260806 22:45:02.792322 26133 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.001s
I20260806 22:45:02.794030 26160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:02.794476 26133 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:45:02.794667 26133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056275803658-18615-0/minicluster-data/master-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056275803658-18615-0/minicluster-data/master-0/wal
uuid: "045409b588cb43ba85aae707500d8bd7"
format_stamp: "Formatted at 2026-08-06 22:45:02 on dist-test-slave-h3ft"
server_key: "869b9edd1d47da866f51686758046a91"
server_key_iv: "1c330af84a04cd528630affeb9b99920"
server_key_version: "encryptionkey@0"
I20260806 22:45:02.794803 26133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056275803658-18615-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056275803658-18615-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056275803658-18615-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:45:02.828178 26133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:45:02.828859 26133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:45:02.829066 26133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:45:02.837949 26133 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.254:46119
I20260806 22:45:02.838116 26237 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.254:46119 every 8 connection(s)
I20260806 22:45:02.838896 26133 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
I20260806 22:45:02.839932 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 26133
I20260806 22:45:02.840138 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786056275803658-18615-0/minicluster-data/master-0/wal/instance
I20260806 22:45:02.840488 26238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:45:02.840521 18615 external_mini_cluster.cc:1428] Setting key acb1b4f7376df0ac457b424d722e40bb
I20260806 22:45:02.845862 26238 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 045409b588cb43ba85aae707500d8bd7: Bootstrap starting.
I20260806 22:45:02.847666 26238 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 045409b588cb43ba85aae707500d8bd7: Neither blocks nor log segments found. Creating new log.
I20260806 22:45:02.848352 26238 log.cc:824] T 00000000000000000000000000000000 P 045409b588cb43ba85aae707500d8bd7: Log is configured to *not* fsync() on all Append() calls
I20260806 22:45:02.850272 26238 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 045409b588cb43ba85aae707500d8bd7: No bootstrap required, opened a new log
I20260806 22:45:02.856737 26238 raft_consensus.cc:348] T 00000000000000000000000000000000 P 045409b588cb43ba85aae707500d8bd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "045409b588cb43ba85aae707500d8bd7" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 46119 } }
I20260806 22:45:02.857259 26238 raft_consensus.cc:374] T 00000000000000000000000000000000 P 045409b588cb43ba85aae707500d8bd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:45:02.857380 26238 raft_consensus.cc:729] T 00000000000000000000000000000000 P 045409b588cb43ba85aae707500d8bd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 045409b588cb43ba85aae707500d8bd7, State: Initialized, Role: FOLLOWER
I20260806 22:45:02.857767 26238 consensus_queue.cc:260] T 00000000000000000000000000000000 P 045409b588cb43ba85aae707500d8bd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "045409b588cb43ba85aae707500d8bd7" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 46119 } }
I20260806 22:45:02.858001 26238 raft_consensus.cc:388] T 00000000000000000000000000000000 P 045409b588cb43ba85aae707500d8bd7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:45:02.858120 26238 raft_consensus.cc:482] T 00000000000000000000000000000000 P 045409b588cb43ba85aae707500d8bd7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:45:02.858264 26238 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 045409b588cb43ba85aae707500d8bd7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:45:02.858973 26244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:02.842095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42152 (local address 127.18.45.254:46119)
0806 22:45:02.846358 (+ 4263us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:02.846374 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:02.846454 (+ 80us) server_negotiation.cc:407] Connection header received
0806 22:45:02.847982 (+ 1528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:02.847999 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:02.848208 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:02.848685 (+ 477us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:02.849298 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:02.851224 (+ 1926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:02.852842 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:02.854243 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:02.854568 (+ 325us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:02.856509 (+ 1941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:02.856518 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:02.856519 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:45:02.857060 (+ 541us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:02.857225 (+ 165us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:45:02.857421 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:02.857667 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:02.858249 (+ 582us) server_negotiation.cc:299] Negotiation successful
0806 22:45:02.858479 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4082,"thread_start_us":156,"threads_started":1}
I20260806 22:45:02.859862 26238 raft_consensus.cc:504] T 00000000000000000000000000000000 P 045409b588cb43ba85aae707500d8bd7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "045409b588cb43ba85aae707500d8bd7" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 46119 } }
I20260806 22:45:02.860026 26238 leader_election.cc:302] T 00000000000000000000000000000000 P 045409b588cb43ba85aae707500d8bd7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 045409b588cb43ba85aae707500d8bd7; no voters:
I20260806 22:45:02.860457 26238 leader_election.cc:288] T 00000000000000000000000000000000 P 045409b588cb43ba85aae707500d8bd7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:45:02.861090 26238 sys_catalog.cc:564] T 00000000000000000000000000000000 P 045409b588cb43ba85aae707500d8bd7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:45:02.861284 18615 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:45:02.861105 26247 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 045409b588cb43ba85aae707500d8bd7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:45:02.861613 26247 raft_consensus.cc:686] T 00000000000000000000000000000000 P 045409b588cb43ba85aae707500d8bd7 [term 1 LEADER]: Becoming Leader. State: Replica: 045409b588cb43ba85aae707500d8bd7, State: Running, Role: LEADER
I20260806 22:45:02.861820 26247 consensus_queue.cc:237] T 00000000000000000000000000000000 P 045409b588cb43ba85aae707500d8bd7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "045409b588cb43ba85aae707500d8bd7" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 46119 } }
I20260806 22:45:02.862640 26248 sys_catalog.cc:455] T 00000000000000000000000000000000 P 045409b588cb43ba85aae707500d8bd7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "045409b588cb43ba85aae707500d8bd7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "045409b588cb43ba85aae707500d8bd7" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 46119 } } }
I20260806 22:45:02.862800 26248 sys_catalog.cc:458] T 00000000000000000000000000000000 P 045409b588cb43ba85aae707500d8bd7 [sys.catalog]: This master's current role is: LEADER
I20260806 22:45:02.863068 26248 sys_catalog.cc:455] T 00000000000000000000000000000000 P 045409b588cb43ba85aae707500d8bd7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 045409b588cb43ba85aae707500d8bd7. Latest consensus state: current_term: 1 leader_uuid: "045409b588cb43ba85aae707500d8bd7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "045409b588cb43ba85aae707500d8bd7" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 46119 } } }
I20260806 22:45:02.863173 26248 sys_catalog.cc:458] T 00000000000000000000000000000000 P 045409b588cb43ba85aae707500d8bd7 [sys.catalog]: This master's current role is: LEADER
W20260806 22:45:02.865643 26273 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 045409b588cb43ba85aae707500d8bd7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:45:02.865763 26273 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:45:02.866003 26262 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:45:02.866410 26262 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:45:02.869606 26262 catalog_manager.cc:1347] Generated new cluster ID: 6a96cde164214c2192d0bf14a282f2d2
I20260806 22:45:02.869722 26262 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:45:02.888499 26262 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:45:02.889950 26262 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:45:02.906136 26262 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 045409b588cb43ba85aae707500d8bd7: Generated new TSK 0
I20260806 22:45:02.906802 26262 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 22:45:02.953297 26287 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260806 22:45:02.953513 26287 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 22:45:02.954356 26244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:02.931496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42166 (local address 127.18.45.254:46119)
0806 22:45:02.931609 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:02.931612 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:02.931799 (+ 187us) server_negotiation.cc:407] Connection header received
0806 22:45:02.931992 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:02.931993 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:02.932022 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:02.932195 (+ 173us) server_negotiation.cc:226] Negotiated authn=JWT
0806 22:45:02.932526 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:02.933836 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:02.935368 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:02.935886 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:02.936028 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:02.936390 (+ 362us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 22:45:02.953794 (+ 17404us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40491/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 22:45:02.954163 (+ 369us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42166: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40491/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":55}
W20260806 22:45:02.954423 26286 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:45:02.931342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:46119 (local address 127.0.0.1:42166)
0806 22:45:02.931679 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:02.931700 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:02.931798 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:02.932220 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:02.932221 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:02.932224 (+ 3us) client_negotiation.cc:189] Negotiated authn=JWT
0806 22:45:02.932346 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:02.932357 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:02.934002 (+ 1645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:02.934004 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:02.935251 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:02.935262 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:02.936064 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:02.936066 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:02.936167 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:02.936176 (+ 9us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 22:45:02.954046 (+ 17870us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40491/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 22:45:02.954268 (+ 222us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.18.45.254:46119: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40491/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":230,"threads_started":1}
W20260806 22:45:02.954476 26244 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42166: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40491/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
I20260806 22:45:02.955298 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 26133
2026-08-06T22:45:03Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (455 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260806 22:45:03.019336 18615 mini_oidc.cc:152] Starting JWKS server
I20260806 22:45:03.020674 18615 webserver.cc:460] Webserver started at https://127.0.0.1:44147/ using document root <none> and password file <none>
I20260806 22:45:03.020761 18615 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 22:45:03.021072 18615 webserver.cc:460] Webserver started at http://127.0.0.1:33409/ using document root <none> and password file <none>
2026-08-06T22:45:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:45:03Z Disabled control of system clock
I20260806 22:45:03.039812 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786056275803658-18615-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786056275803658-18615-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786056275803658-18615-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:38455
--webserver_interface=127.18.45.254
--webserver_port=0
--builtin_ntp_servers=127.18.45.212:41601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.45.254:38455
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44147/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 22:45:03.202692 26295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:45:03.203260 26295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:45:03.203313 26295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:45:03.205597 26295 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:45:03.205706 26295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:45:03.205724 26295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:45:03.205747 26295 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:45:03.205775 26295 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 22:45:03.205813 26295 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 22:45:03.205835 26295 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44147/jwks.json
I20260806 22:45:03.208521 26295 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:41601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786056275803658-18615-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786056275803658-18615-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.45.254:38455
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:38455
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44147/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.45.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26295
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786056275803658-18615-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:45:03.209044 26295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:45:03.209566 26295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:45:03.219146 26303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:03.219121 26302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:03.219179 26307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:03.219726 26295 server_base.cc:1034] running on GCE node
I20260806 22:45:03.220321 26295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:45:03.220956 26295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:45:03.222215 26295 hybrid_clock.cc:648] HybridClock initialized: now 1786056303222190 us; error 69 us; skew 500 ppm
I20260806 22:45:03.225014 26295 webserver.cc:460] Webserver started at http://127.18.45.254:35793/ using document root <none> and password file <none>
I20260806 22:45:03.225574 26295 fs_manager.cc:359] Metadata directory not provided
I20260806 22:45:03.225705 26295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:45:03.226006 26295 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:45:03.228334 26295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786056275803658-18615-0/minicluster-data/master-0/data/instance:
uuid: "f4821958c3474e3a8c591b6fb7d6d939"
format_stamp: "Formatted at 2026-08-06 22:45:03 on dist-test-slave-h3ft"
server_key: "3225a463e710585ad43d3f6ee61ecef5"
server_key_iv: "fd67a56394963a5491eadf6ceccb4a7e"
server_key_version: "encryptionkey@0"
I20260806 22:45:03.229143 26295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786056275803658-18615-0/minicluster-data/master-0/wal/instance:
uuid: "f4821958c3474e3a8c591b6fb7d6d939"
format_stamp: "Formatted at 2026-08-06 22:45:03 on dist-test-slave-h3ft"
server_key: "3225a463e710585ad43d3f6ee61ecef5"
server_key_iv: "fd67a56394963a5491eadf6ceccb4a7e"
server_key_version: "encryptionkey@0"
I20260806 22:45:03.232436 26295 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 22:45:03.234225 26315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:03.234627 26295 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260806 22:45:03.234838 26295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786056275803658-18615-0/minicluster-data/master-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786056275803658-18615-0/minicluster-data/master-0/wal
uuid: "f4821958c3474e3a8c591b6fb7d6d939"
format_stamp: "Formatted at 2026-08-06 22:45:03 on dist-test-slave-h3ft"
server_key: "3225a463e710585ad43d3f6ee61ecef5"
server_key_iv: "fd67a56394963a5491eadf6ceccb4a7e"
server_key_version: "encryptionkey@0"
I20260806 22:45:03.235056 26295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786056275803658-18615-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786056275803658-18615-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786056275803658-18615-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:45:03.277379 26295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:45:03.278141 26295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:45:03.278451 26295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:45:03.287499 26295 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.254:38455
I20260806 22:45:03.287602 26390 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.254:38455 every 8 connection(s)
I20260806 22:45:03.288311 26295 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
I20260806 22:45:03.289794 26391 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:45:03.292311 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 26295
I20260806 22:45:03.292550 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786056275803658-18615-0/minicluster-data/master-0/wal/instance
I20260806 22:45:03.292953 18615 external_mini_cluster.cc:1428] Setting key 180f8e49cd3a7270fe171544cc34e4df
I20260806 22:45:03.297291 26391 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4821958c3474e3a8c591b6fb7d6d939: Bootstrap starting.
I20260806 22:45:03.298789 26391 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f4821958c3474e3a8c591b6fb7d6d939: Neither blocks nor log segments found. Creating new log.
I20260806 22:45:03.299561 26391 log.cc:824] T 00000000000000000000000000000000 P f4821958c3474e3a8c591b6fb7d6d939: Log is configured to *not* fsync() on all Append() calls
I20260806 22:45:03.301092 26391 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4821958c3474e3a8c591b6fb7d6d939: No bootstrap required, opened a new log
I20260806 22:45:03.304141 26394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:03.294153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35472 (local address 127.18.45.254:38455)
0806 22:45:03.294546 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:03.294567 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:03.294623 (+ 56us) server_negotiation.cc:407] Connection header received
0806 22:45:03.295561 (+ 938us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:03.295578 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:03.295706 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:03.296069 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:03.296543 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:03.298211 (+ 1668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:03.299807 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:03.300840 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:03.301151 (+ 311us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:03.302618 (+ 1467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:03.302632 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:03.302632 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:45:03.302972 (+ 340us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:03.303116 (+ 144us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:45:03.303142 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:03.303300 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:03.303507 (+ 207us) server_negotiation.cc:299] Negotiation successful
0806 22:45:03.303712 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":154,"threads_started":1}
I20260806 22:45:03.305869 18615 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:45:03.305593 26391 raft_consensus.cc:348] T 00000000000000000000000000000000 P f4821958c3474e3a8c591b6fb7d6d939 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4821958c3474e3a8c591b6fb7d6d939" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 38455 } }
I20260806 22:45:03.305977 26391 raft_consensus.cc:374] T 00000000000000000000000000000000 P f4821958c3474e3a8c591b6fb7d6d939 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:45:03.306102 26391 raft_consensus.cc:729] T 00000000000000000000000000000000 P f4821958c3474e3a8c591b6fb7d6d939 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4821958c3474e3a8c591b6fb7d6d939, State: Initialized, Role: FOLLOWER
I20260806 22:45:03.306371 26391 consensus_queue.cc:260] T 00000000000000000000000000000000 P f4821958c3474e3a8c591b6fb7d6d939 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4821958c3474e3a8c591b6fb7d6d939" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 38455 } }
I20260806 22:45:03.306524 26391 raft_consensus.cc:388] T 00000000000000000000000000000000 P f4821958c3474e3a8c591b6fb7d6d939 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:45:03.306641 26391 raft_consensus.cc:482] T 00000000000000000000000000000000 P f4821958c3474e3a8c591b6fb7d6d939 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:45:03.306751 26391 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f4821958c3474e3a8c591b6fb7d6d939 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:45:03.308231 26391 raft_consensus.cc:504] T 00000000000000000000000000000000 P f4821958c3474e3a8c591b6fb7d6d939 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4821958c3474e3a8c591b6fb7d6d939" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 38455 } }
I20260806 22:45:03.308444 26391 leader_election.cc:302] T 00000000000000000000000000000000 P f4821958c3474e3a8c591b6fb7d6d939 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f4821958c3474e3a8c591b6fb7d6d939; no voters:
I20260806 22:45:03.308779 26391 leader_election.cc:288] T 00000000000000000000000000000000 P f4821958c3474e3a8c591b6fb7d6d939 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:45:03.308951 26396 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f4821958c3474e3a8c591b6fb7d6d939 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:45:03.309226 26396 raft_consensus.cc:686] T 00000000000000000000000000000000 P f4821958c3474e3a8c591b6fb7d6d939 [term 1 LEADER]: Becoming Leader. State: Replica: f4821958c3474e3a8c591b6fb7d6d939, State: Running, Role: LEADER
I20260806 22:45:03.309396 26391 sys_catalog.cc:564] T 00000000000000000000000000000000 P f4821958c3474e3a8c591b6fb7d6d939 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:45:03.309409 26396 consensus_queue.cc:237] T 00000000000000000000000000000000 P f4821958c3474e3a8c591b6fb7d6d939 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4821958c3474e3a8c591b6fb7d6d939" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 38455 } }
I20260806 22:45:03.310261 26398 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4821958c3474e3a8c591b6fb7d6d939 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f4821958c3474e3a8c591b6fb7d6d939. Latest consensus state: current_term: 1 leader_uuid: "f4821958c3474e3a8c591b6fb7d6d939" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4821958c3474e3a8c591b6fb7d6d939" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 38455 } } }
I20260806 22:45:03.310477 26398 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4821958c3474e3a8c591b6fb7d6d939 [sys.catalog]: This master's current role is: LEADER
I20260806 22:45:03.310810 26397 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4821958c3474e3a8c591b6fb7d6d939 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f4821958c3474e3a8c591b6fb7d6d939" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4821958c3474e3a8c591b6fb7d6d939" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 38455 } } }
I20260806 22:45:03.310933 26397 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4821958c3474e3a8c591b6fb7d6d939 [sys.catalog]: This master's current role is: LEADER
I20260806 22:45:03.311662 26405 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:45:03.312883 26405 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:45:03.315821 26405 catalog_manager.cc:1347] Generated new cluster ID: a9cb938c5eb44f15bcee621f0fe4d46a
I20260806 22:45:03.315973 26405 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:45:03.330518 26405 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:45:03.332001 26405 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:45:03.352147 26405 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f4821958c3474e3a8c591b6fb7d6d939: Generated new TSK 0
I20260806 22:45:03.352663 26405 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:45:03.372938 26394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:03.365545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35478 (local address 127.18.45.254:38455)
0806 22:45:03.365621 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:03.365624 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:03.365754 (+ 130us) server_negotiation.cc:407] Connection header received
0806 22:45:03.365891 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:03.365892 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:03.365928 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:03.366113 (+ 185us) server_negotiation.cc:226] Negotiated authn=JWT
0806 22:45:03.366305 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:03.367713 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:03.369259 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:03.369787 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:03.369905 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:03.370180 (+ 275us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 22:45:03.372474 (+ 2294us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44147/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 22:45:03.372801 (+ 327us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35478: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44147/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":39}
W20260806 22:45:03.373090 26394 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35478: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44147/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260806 22:45:03.375339 26431 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:45:03.365414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:38455 (local address 127.0.0.1:35478)
0806 22:45:03.365631 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:03.365657 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:03.365763 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:03.366107 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:03.366108 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:03.366111 (+ 3us) client_negotiation.cc:189] Negotiated authn=JWT
0806 22:45:03.366213 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:03.366217 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:03.367867 (+ 1650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:03.367871 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:03.369119 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:03.369124 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:03.369917 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:03.369918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:03.369983 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:03.369990 (+ 7us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 22:45:03.372671 (+ 2681us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44147/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 22:45:03.375170 (+ 2499us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.18.45.254:38455: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44147/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":180,"mutex_wait_us":2062,"thread_start_us":153,"threads_started":1}
I20260806 22:45:03.376011 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 26295
E20260806 22:45:03.404866 26432 webserver.cc:533] Webserver: sslize failed:
E20260806 22:45:03.405093 26432 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-06T22:45:03Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (422 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-06T22:45:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:45:03Z Disabled control of system clock
I20260806 22:45:03.458510 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056275803658-18615-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056275803658-18615-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056275803658-18615-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:38231
--webserver_interface=127.18.45.254
--webserver_port=0
--builtin_ntp_servers=127.18.45.212:40839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.45.254:38231
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056275803658-18615-0/testchainca.cert with env {}
W20260806 22:45:03.600201 26437 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:45:03.600730 26437 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:45:03.600796 26437 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:45:03.602926 26437 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:45:03.603101 26437 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:45:03.603191 26437 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:45:03.603261 26437 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:45:03.603308 26437 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 22:45:03.603369 26437 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 22:45:03.603415 26437 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260806 22:45:03.606529 26437 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:40839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056275803658-18615-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056275803658-18615-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.45.254:38231
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:38231
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.45.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056275803658-18615-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26437
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056275803658-18615-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:45:03.607136 26437 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:45:03.607592 26437 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:45:03.616035 26448 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:03.616035 26454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:03.616061 26447 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:03.616739 26437 server_base.cc:1034] running on GCE node
I20260806 22:45:03.617214 26437 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:45:03.617786 26437 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:45:03.619127 26437 hybrid_clock.cc:648] HybridClock initialized: now 1786056303619092 us; error 62 us; skew 500 ppm
I20260806 22:45:03.621559 26437 webserver.cc:460] Webserver started at http://127.18.45.254:38351/ using document root <none> and password file <none>
I20260806 22:45:03.621999 26437 fs_manager.cc:359] Metadata directory not provided
I20260806 22:45:03.622135 26437 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:45:03.622407 26437 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:45:03.624415 26437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056275803658-18615-0/minicluster-data/master-0/data/instance:
uuid: "d0a79b30f6c445d98c66fd1bf4e8c2f2"
format_stamp: "Formatted at 2026-08-06 22:45:03 on dist-test-slave-h3ft"
server_key: "bf766b820fc25a12bf4e79f0c695ad15"
server_key_iv: "366e3bab2908ca911f9eee442667a0cd"
server_key_version: "encryptionkey@0"
I20260806 22:45:03.625097 26437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056275803658-18615-0/minicluster-data/master-0/wal/instance:
uuid: "d0a79b30f6c445d98c66fd1bf4e8c2f2"
format_stamp: "Formatted at 2026-08-06 22:45:03 on dist-test-slave-h3ft"
server_key: "bf766b820fc25a12bf4e79f0c695ad15"
server_key_iv: "366e3bab2908ca911f9eee442667a0cd"
server_key_version: "encryptionkey@0"
I20260806 22:45:03.627950 26437 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260806 22:45:03.629429 26463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:03.629719 26437 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:45:03.629946 26437 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056275803658-18615-0/minicluster-data/master-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056275803658-18615-0/minicluster-data/master-0/wal
uuid: "d0a79b30f6c445d98c66fd1bf4e8c2f2"
format_stamp: "Formatted at 2026-08-06 22:45:03 on dist-test-slave-h3ft"
server_key: "bf766b820fc25a12bf4e79f0c695ad15"
server_key_iv: "366e3bab2908ca911f9eee442667a0cd"
server_key_version: "encryptionkey@0"
I20260806 22:45:03.630081 26437 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056275803658-18615-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056275803658-18615-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056275803658-18615-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:45:03.649164 26437 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:45:03.650024 26437 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:45:03.650375 26437 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:45:03.661015 26437 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.254:38231
I20260806 22:45:03.661129 26544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.254:38231 every 8 connection(s)
I20260806 22:45:03.662042 26437 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
I20260806 22:45:03.664569 26546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:45:03.669461 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 26437
I20260806 22:45:03.669706 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786056275803658-18615-0/minicluster-data/master-0/wal/instance
I20260806 22:45:03.670089 18615 external_mini_cluster.cc:1428] Setting key 955c41a825e87038956453daecbf873f
I20260806 22:45:03.670581 26546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0a79b30f6c445d98c66fd1bf4e8c2f2: Bootstrap starting.
I20260806 22:45:03.672292 26546 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d0a79b30f6c445d98c66fd1bf4e8c2f2: Neither blocks nor log segments found. Creating new log.
I20260806 22:45:03.672920 26546 log.cc:824] T 00000000000000000000000000000000 P d0a79b30f6c445d98c66fd1bf4e8c2f2: Log is configured to *not* fsync() on all Append() calls
I20260806 22:45:03.674310 26546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0a79b30f6c445d98c66fd1bf4e8c2f2: No bootstrap required, opened a new log
I20260806 22:45:03.677925 26546 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0a79b30f6c445d98c66fd1bf4e8c2f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0a79b30f6c445d98c66fd1bf4e8c2f2" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 38231 } }
I20260806 22:45:03.678328 26546 raft_consensus.cc:374] T 00000000000000000000000000000000 P d0a79b30f6c445d98c66fd1bf4e8c2f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:45:03.678414 26546 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0a79b30f6c445d98c66fd1bf4e8c2f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0a79b30f6c445d98c66fd1bf4e8c2f2, State: Initialized, Role: FOLLOWER
I20260806 22:45:03.678692 26546 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0a79b30f6c445d98c66fd1bf4e8c2f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0a79b30f6c445d98c66fd1bf4e8c2f2" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 38231 } }
I20260806 22:45:03.678943 26546 raft_consensus.cc:388] T 00000000000000000000000000000000 P d0a79b30f6c445d98c66fd1bf4e8c2f2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:45:03.679171 26546 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0a79b30f6c445d98c66fd1bf4e8c2f2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:45:03.679279 26546 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0a79b30f6c445d98c66fd1bf4e8c2f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:45:03.680974 26554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:03.671217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38418 (local address 127.18.45.254:38231)
0806 22:45:03.671655 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:03.671666 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:03.671720 (+ 54us) server_negotiation.cc:407] Connection header received
0806 22:45:03.672679 (+ 959us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:03.672698 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:03.672802 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:03.673101 (+ 299us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:03.673514 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:03.675161 (+ 1647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:03.676515 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:03.677319 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:03.677613 (+ 294us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:03.679232 (+ 1619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:03.679248 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:03.679248 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:45:03.679746 (+ 498us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:03.679858 (+ 112us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:45:03.679900 (+ 42us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:03.680060 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:03.680177 (+ 117us) server_negotiation.cc:299] Negotiation successful
0806 22:45:03.680399 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":189,"threads_started":1}
I20260806 22:45:03.681200 26546 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0a79b30f6c445d98c66fd1bf4e8c2f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0a79b30f6c445d98c66fd1bf4e8c2f2" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 38231 } }
I20260806 22:45:03.681394 26546 leader_election.cc:302] T 00000000000000000000000000000000 P d0a79b30f6c445d98c66fd1bf4e8c2f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d0a79b30f6c445d98c66fd1bf4e8c2f2; no voters:
I20260806 22:45:03.681939 26557 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0a79b30f6c445d98c66fd1bf4e8c2f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:45:03.681928 26546 leader_election.cc:288] T 00000000000000000000000000000000 P d0a79b30f6c445d98c66fd1bf4e8c2f2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:45:03.682325 26557 raft_consensus.cc:686] T 00000000000000000000000000000000 P d0a79b30f6c445d98c66fd1bf4e8c2f2 [term 1 LEADER]: Becoming Leader. State: Replica: d0a79b30f6c445d98c66fd1bf4e8c2f2, State: Running, Role: LEADER
I20260806 22:45:03.682533 26557 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0a79b30f6c445d98c66fd1bf4e8c2f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0a79b30f6c445d98c66fd1bf4e8c2f2" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 38231 } }
I20260806 22:45:03.682704 26546 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0a79b30f6c445d98c66fd1bf4e8c2f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:45:03.683316 26557 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0a79b30f6c445d98c66fd1bf4e8c2f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d0a79b30f6c445d98c66fd1bf4e8c2f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0a79b30f6c445d98c66fd1bf4e8c2f2" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 38231 } } }
I20260806 22:45:03.683466 26557 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0a79b30f6c445d98c66fd1bf4e8c2f2 [sys.catalog]: This master's current role is: LEADER
I20260806 22:45:03.683580 18615 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:45:03.684064 26566 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:45:03.684278 26558 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0a79b30f6c445d98c66fd1bf4e8c2f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0a79b30f6c445d98c66fd1bf4e8c2f2. Latest consensus state: current_term: 1 leader_uuid: "d0a79b30f6c445d98c66fd1bf4e8c2f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0a79b30f6c445d98c66fd1bf4e8c2f2" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 38231 } } }
I20260806 22:45:03.684432 26558 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0a79b30f6c445d98c66fd1bf4e8c2f2 [sys.catalog]: This master's current role is: LEADER
I20260806 22:45:03.685413 26566 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:45:03.690291 26566 catalog_manager.cc:1347] Generated new cluster ID: e1bcbf2e8881402b9827689f28c0332f
I20260806 22:45:03.690433 26566 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:45:03.703701 26566 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:45:03.705003 26566 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:45:03.710332 26566 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d0a79b30f6c445d98c66fd1bf4e8c2f2: Generated new TSK 0
I20260806 22:45:03.710811 26566 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:45:03.752537 26554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:03.727373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38430 (local address 127.18.45.254:38231)
0806 22:45:03.727505 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:03.727508 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:03.727775 (+ 267us) server_negotiation.cc:407] Connection header received
0806 22:45:03.727927 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:03.727929 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:03.727962 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:03.728181 (+ 219us) server_negotiation.cc:226] Negotiated authn=JWT
0806 22:45:03.728384 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:03.729676 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:03.731268 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:03.731799 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:03.731966 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:03.732324 (+ 358us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 22:45:03.751940 (+ 19616us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 22:45:03.752394 (+ 454us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38430: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":67}
W20260806 22:45:03.752614 26601 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:45:03.727244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:38231 (local address 127.0.0.1:38430)
0806 22:45:03.727631 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:03.727651 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:03.727783 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:03.728165 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:03.728174 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:03.728177 (+ 3us) client_negotiation.cc:189] Negotiated authn=JWT
0806 22:45:03.728282 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:03.728286 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:03.729967 (+ 1681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:03.729968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:03.731113 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:03.731119 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:03.732007 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:03.732009 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:03.732096 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:03.732105 (+ 9us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 22:45:03.752257 (+ 20152us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 22:45:03.752478 (+ 221us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.18.45.254:38231: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":259,"threads_started":1}
W20260806 22:45:03.752725 26554 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38430: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
I20260806 22:45:03.753686 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 26437
2026-08-06T22:45:03Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (347 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (171 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (163 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:45:04 dist-test-slave-h3ft krb5kdc[26615](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:45:04 dist-test-slave-h3ft krb5kdc[26615](info): set up 2 sockets
Aug 06 22:45:04 dist-test-slave-h3ft krb5kdc[26615](info): commencing operation
krb5kdc: starting...
W20260806 22:45:05.702256 18615 mini_kdc.cc:121] Time spent starting KDC: real 1.576s user 0.000s sys 0.012s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Aug 06 22:45:05 dist-test-slave-h3ft krb5kdc[26615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056305, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:45:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:45:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.45.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab.
Entry for principal kudu/127.18.45.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab.
I20260806 22:45:05.862900 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:38509
--webserver_interface=127.18.45.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:44455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.45.254:38509
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:45:05.964366 26634 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:45:05.964710 26634 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:45:05.964752 26634 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:45:05.966092 26634 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:45:05.966171 26634 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:45:05.966197 26634 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:45:05.966225 26634 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:45:05.966254 26634 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:45:05.967926 26634 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:44455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.45.254:38509
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:38509
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.45.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26634
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:45:05.968250 26634 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:45:05.968540 26634 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:45:05.974231 26641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:05.974241 26646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:05.974267 26642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:05.974579 26634 server_base.cc:1034] running on GCE node
I20260806 22:45:05.974921 26634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:45:05.975329 26634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:45:05.976500 26634 hybrid_clock.cc:648] HybridClock initialized: now 1786056305976465 us; error 49 us; skew 500 ppm
Aug 06 22:45:05 dist-test-slave-h3ft krb5kdc[26615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056305, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:45:05.984407 26634 init.cc:377] Logged in from keytab as kudu/127.18.45.254@KRBTEST.COM (short username kudu)
I20260806 22:45:05.985064 26634 webserver.cc:460] Webserver started at http://127.18.45.254:45753/ using document root <none> and password file <none>
I20260806 22:45:05.985394 26634 fs_manager.cc:359] Metadata directory not provided
I20260806 22:45:05.985471 26634 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:45:05.985642 26634 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:45:05.987097 26634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/minicluster-data/master-0/data/instance:
uuid: "1897d7dfd1d24d20801962371e88ad9c"
format_stamp: "Formatted at 2026-08-06 22:45:05 on dist-test-slave-h3ft"
server_key: "7285ab7cf9d088d26077a46a208766f4"
server_key_iv: "bbde4b1ef8f44b6555e6666acd0da3fc"
server_key_version: "encryptionkey@0"
I20260806 22:45:05.987562 26634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/minicluster-data/master-0/wal/instance:
uuid: "1897d7dfd1d24d20801962371e88ad9c"
format_stamp: "Formatted at 2026-08-06 22:45:05 on dist-test-slave-h3ft"
server_key: "7285ab7cf9d088d26077a46a208766f4"
server_key_iv: "bbde4b1ef8f44b6555e6666acd0da3fc"
server_key_version: "encryptionkey@0"
I20260806 22:45:05.989331 26634 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.001s
I20260806 22:45:05.990403 26654 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:05.990669 26634 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:45:05.990790 26634 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/minicluster-data/master-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/minicluster-data/master-0/wal
uuid: "1897d7dfd1d24d20801962371e88ad9c"
format_stamp: "Formatted at 2026-08-06 22:45:05 on dist-test-slave-h3ft"
server_key: "7285ab7cf9d088d26077a46a208766f4"
server_key_iv: "bbde4b1ef8f44b6555e6666acd0da3fc"
server_key_version: "encryptionkey@0"
I20260806 22:45:05.990881 26634 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:45:06.011763 26634 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:45:06.014777 26634 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:45:06.015128 26634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:45:06.020314 26634 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.254:38509
I20260806 22:45:06.020396 26741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.254:38509 every 8 connection(s)
I20260806 22:45:06.020845 26634 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
I20260806 22:45:06.021944 26742 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:45:06.024787 26742 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1897d7dfd1d24d20801962371e88ad9c: Bootstrap starting.
I20260806 22:45:06.025449 26742 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1897d7dfd1d24d20801962371e88ad9c: Neither blocks nor log segments found. Creating new log.
I20260806 22:45:06.025758 26742 log.cc:824] T 00000000000000000000000000000000 P 1897d7dfd1d24d20801962371e88ad9c: Log is configured to *not* fsync() on all Append() calls
I20260806 22:45:06.026577 26742 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1897d7dfd1d24d20801962371e88ad9c: No bootstrap required, opened a new log
I20260806 22:45:06.028457 26742 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1897d7dfd1d24d20801962371e88ad9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1897d7dfd1d24d20801962371e88ad9c" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 38509 } }
I20260806 22:45:06.028622 26742 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1897d7dfd1d24d20801962371e88ad9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:45:06.028647 26742 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1897d7dfd1d24d20801962371e88ad9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1897d7dfd1d24d20801962371e88ad9c, State: Initialized, Role: FOLLOWER
I20260806 22:45:06.028812 26742 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1897d7dfd1d24d20801962371e88ad9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1897d7dfd1d24d20801962371e88ad9c" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 38509 } }
I20260806 22:45:06.028901 26742 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1897d7dfd1d24d20801962371e88ad9c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:45:06.028928 26742 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1897d7dfd1d24d20801962371e88ad9c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:45:06.028963 26742 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1897d7dfd1d24d20801962371e88ad9c [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:45:06.029568 26742 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1897d7dfd1d24d20801962371e88ad9c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1897d7dfd1d24d20801962371e88ad9c" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 38509 } }
I20260806 22:45:06.029666 26742 leader_election.cc:302] T 00000000000000000000000000000000 P 1897d7dfd1d24d20801962371e88ad9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1897d7dfd1d24d20801962371e88ad9c; no voters:
I20260806 22:45:06.029816 26742 leader_election.cc:288] T 00000000000000000000000000000000 P 1897d7dfd1d24d20801962371e88ad9c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:45:06.029951 26745 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1897d7dfd1d24d20801962371e88ad9c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:45:06.030128 26742 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1897d7dfd1d24d20801962371e88ad9c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:45:06.030575 26745 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1897d7dfd1d24d20801962371e88ad9c [term 1 LEADER]: Becoming Leader. State: Replica: 1897d7dfd1d24d20801962371e88ad9c, State: Running, Role: LEADER
I20260806 22:45:06.030721 26745 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1897d7dfd1d24d20801962371e88ad9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1897d7dfd1d24d20801962371e88ad9c" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 38509 } }
I20260806 22:45:06.030947 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 26634
I20260806 22:45:06.031139 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/minicluster-data/master-0/wal/instance
I20260806 22:45:06.031473 18615 external_mini_cluster.cc:1428] Setting key 58af8156d3faa2f84a5d8e400aad4cde
I20260806 22:45:06.031450 26746 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1897d7dfd1d24d20801962371e88ad9c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1897d7dfd1d24d20801962371e88ad9c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1897d7dfd1d24d20801962371e88ad9c" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 38509 } } }
I20260806 22:45:06.031538 26746 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1897d7dfd1d24d20801962371e88ad9c [sys.catalog]: This master's current role is: LEADER
I20260806 22:45:06.031998 26747 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1897d7dfd1d24d20801962371e88ad9c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1897d7dfd1d24d20801962371e88ad9c. Latest consensus state: current_term: 1 leader_uuid: "1897d7dfd1d24d20801962371e88ad9c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1897d7dfd1d24d20801962371e88ad9c" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 38509 } } }
W20260806 22:45:06.032351 26764 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1897d7dfd1d24d20801962371e88ad9c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260806 22:45:06.032681 26747 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1897d7dfd1d24d20801962371e88ad9c [sys.catalog]: This master's current role is: LEADER
W20260806 22:45:06.032747 26764 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:45:06.032815 26765 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:45:06.032996 26765 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:45:06.034650 26765 catalog_manager.cc:1347] Generated new cluster ID: af4e3392ec5d4ab29df65abf05f54678
I20260806 22:45:06.034715 26765 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:45:06 dist-test-slave-h3ft krb5kdc[26615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056305, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:45:06.043867 26769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:06.032355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48670 (local address 127.18.45.254:38509)
0806 22:45:06.032637 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:06.032648 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:06.032690 (+ 42us) server_negotiation.cc:407] Connection header received
0806 22:45:06.032829 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:06.032837 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:06.032903 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:06.033086 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:06.034090 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:06.035330 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:06.036176 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:06.036620 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:06.036741 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:06.039479 (+ 2738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:06.039485 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:06.039486 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:06.039510 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:06.042032 (+ 2522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:06.042452 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:06.042456 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:06.042457 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:06.042572 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:06.042799 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:06.042800 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:06.042801 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:06.043166 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:06.043371 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:06.043465 (+ 94us) server_negotiation.cc:299] Negotiation successful
0806 22:45:06.043634 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":97,"threads_started":1}
I20260806 22:45:06.050696 26765 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:45:06.051412 26765 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:45:06.059389 26765 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1897d7dfd1d24d20801962371e88ad9c: Generated new TSK 0
I20260806 22:45:06.059646 26765 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.45.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab.
Entry for principal kudu/127.18.45.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab.
I20260806 22:45:06.098754 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.193:0
--local_ip_for_outbound_sockets=127.18.45.193
--webserver_interface=127.18.45.193
--webserver_port=0
--tserver_master_addrs=127.18.45.254:38509
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab
--principal=kudu/127.18.45.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:44455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:45:06.199146 26776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:45:06.199414 26776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:45:06.199443 26776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:45:06.200768 26776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:45:06.200820 26776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:45:06.200850 26776 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.193
I20260806 22:45:06.202544 26776 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:44455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab
--principal=kudu/127.18.45.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.45.193
--webserver_port=0
--tserver_master_addrs=127.18.45.254:38509
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26776
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.193
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:45:06.202857 26776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:45:06.203202 26776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:45:06.209424 26786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:06.209400 26783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:06.209465 26776 server_base.cc:1034] running on GCE node
W20260806 22:45:06.209393 26784 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:06.210055 26776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:45:06.210366 26776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:45:06.211548 26776 hybrid_clock.cc:648] HybridClock initialized: now 1786056306211521 us; error 43 us; skew 500 ppm
Aug 06 22:45:06 dist-test-slave-h3ft krb5kdc[26615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056306, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:45:06.214924 26776 init.cc:377] Logged in from keytab as kudu/127.18.45.193@KRBTEST.COM (short username kudu)
I20260806 22:45:06.215827 26776 webserver.cc:460] Webserver started at http://127.18.45.193:46735/ using document root <none> and password file <none>
I20260806 22:45:06.216154 26776 fs_manager.cc:359] Metadata directory not provided
I20260806 22:45:06.216234 26776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:45:06.216401 26776 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:45:06.217749 26776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/minicluster-data/ts-0/data/instance:
uuid: "bf231d6dafd246b2bd6c2c9494d58c03"
format_stamp: "Formatted at 2026-08-06 22:45:06 on dist-test-slave-h3ft"
server_key: "964db20e1808d0769c6ad232f6d768ea"
server_key_iv: "2bef4b7717be4ca259a41c859a8490a6"
server_key_version: "encryptionkey@0"
I20260806 22:45:06.218173 26776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/minicluster-data/ts-0/wal/instance:
uuid: "bf231d6dafd246b2bd6c2c9494d58c03"
format_stamp: "Formatted at 2026-08-06 22:45:06 on dist-test-slave-h3ft"
server_key: "964db20e1808d0769c6ad232f6d768ea"
server_key_iv: "2bef4b7717be4ca259a41c859a8490a6"
server_key_version: "encryptionkey@0"
I20260806 22:45:06.219849 26776 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260806 22:45:06.220737 26795 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:06.220966 26776 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:45:06.221094 26776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/minicluster-data/ts-0/wal
uuid: "bf231d6dafd246b2bd6c2c9494d58c03"
format_stamp: "Formatted at 2026-08-06 22:45:06 on dist-test-slave-h3ft"
server_key: "964db20e1808d0769c6ad232f6d768ea"
server_key_iv: "2bef4b7717be4ca259a41c859a8490a6"
server_key_version: "encryptionkey@0"
I20260806 22:45:06.221176 26776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:45:06.233659 26776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:45:06.236428 26776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:45:06.236620 26776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:45:06.236937 26776 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:45:06.237380 26776 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:45:06.237421 26776 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:06.237450 26776 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:45:06.237463 26776 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:06.245076 26776 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.193:41821
I20260806 22:45:06.245253 26959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.193:41821 every 8 connection(s)
I20260806 22:45:06.245597 26776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
I20260806 22:45:06.247362 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 26776
I20260806 22:45:06.247483 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/minicluster-data/ts-0/wal/instance
I20260806 22:45:06.247722 18615 external_mini_cluster.cc:1428] Setting key bc6798243222fa5cb640f818dcfd42c0
Aug 06 22:45:06 dist-test-slave-h3ft krb5kdc[26615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056306, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.193@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.45.194@KRBTEST.COM; defaulting to no policy
I20260806 22:45:06.259322 26769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:06.246677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:33857 (local address 127.18.45.254:38509)
0806 22:45:06.246759 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:06.246761 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:06.247112 (+ 351us) server_negotiation.cc:407] Connection header received
0806 22:45:06.248556 (+ 1444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:06.248557 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:06.248575 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:06.248619 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:06.249501 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:06.250326 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:06.252076 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:06.252300 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:06.252362 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:06.255776 (+ 3414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:06.255779 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:06.255779 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:06.255795 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:06.257547 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:06.258082 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:06.258084 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:06.258084 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:06.258157 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:06.258462 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:06.258463 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:06.258463 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:06.258616 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:06.258701 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:06.259081 (+ 380us) server_negotiation.cc:299] Negotiation successful
0806 22:45:06.259201 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 22:45:06.259560 26963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:06.246627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:38509 (local address 127.18.45.193:33857)
0806 22:45:06.246914 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:06.246968 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:06.248441 (+ 1473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:06.248739 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:06.248744 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:06.249120 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:06.249380 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:06.249385 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:06.250464 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:06.250466 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:06.251978 (+ 1512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:06.251982 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:06.252399 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:06.252401 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:06.252584 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:06.253313 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:06.253327 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:06.255645 (+ 2318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:45:06.257666 (+ 2021us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:06.257678 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:06.257681 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:06.257986 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:06.258236 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:06.258237 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:06.258238 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:06.258383 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:06.258688 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:45:06.258698 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:45:06.258891 (+ 193us) client_negotiation.cc:769] Sending connection context
0806 22:45:06.259010 (+ 119us) client_negotiation.cc:240] Negotiation successful
0806 22:45:06.259247 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":130,"threads_started":1}
I20260806 22:45:06.260255 26960 heartbeater.cc:347] Connected to a master server at 127.18.45.254:38509
I20260806 22:45:06.260424 26960 heartbeater.cc:464] Registering TS with master...
I20260806 22:45:06.260663 26960 heartbeater.cc:511] Master 127.18.45.254:38509 requested a full tablet report, sending...
I20260806 22:45:06.261427 26684 ts_manager.cc:194] Registered new tserver with Master: bf231d6dafd246b2bd6c2c9494d58c03 (127.18.45.193:41821)
I20260806 22:45:06.262315 26684 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.45.193@KRBTEST.COM'} at 127.18.45.193:33857
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab.
Entry for principal kudu/127.18.45.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab.
I20260806 22:45:06.282411 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.194:0
--local_ip_for_outbound_sockets=127.18.45.194
--webserver_interface=127.18.45.194
--webserver_port=0
--tserver_master_addrs=127.18.45.254:38509
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab
--principal=kudu/127.18.45.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:44455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:45:06.387109 26966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:45:06.387466 26966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:45:06.387516 26966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:45:06.388921 26966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:45:06.389008 26966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:45:06.389062 26966 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.194
I20260806 22:45:06.390950 26966 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:44455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab
--principal=kudu/127.18.45.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.45.194
--webserver_port=0
--tserver_master_addrs=127.18.45.254:38509
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26966
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.194
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:45:06.391359 26966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:45:06.391666 26966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:45:06.398234 26973 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:06.398195 26972 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:06.398195 26975 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:06.398502 26966 server_base.cc:1034] running on GCE node
I20260806 22:45:06.398815 26966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:45:06.399147 26966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:45:06.400326 26966 hybrid_clock.cc:648] HybridClock initialized: now 1786056306400300 us; error 71 us; skew 500 ppm
Aug 06 22:45:06 dist-test-slave-h3ft krb5kdc[26615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056306, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:45:06.403606 26966 init.cc:377] Logged in from keytab as kudu/127.18.45.194@KRBTEST.COM (short username kudu)
I20260806 22:45:06.404141 26966 webserver.cc:460] Webserver started at http://127.18.45.194:46585/ using document root <none> and password file <none>
I20260806 22:45:06.404417 26966 fs_manager.cc:359] Metadata directory not provided
I20260806 22:45:06.404465 26966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:45:06.404574 26966 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:45:06.405921 26966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/minicluster-data/ts-1/data/instance:
uuid: "03489ba1322e410c943615027b79d1c4"
format_stamp: "Formatted at 2026-08-06 22:45:06 on dist-test-slave-h3ft"
server_key: "e6e424020649817b3fdfc64ee0f932aa"
server_key_iv: "0f88360e094d49a2bdb4e3a05b795509"
server_key_version: "encryptionkey@0"
I20260806 22:45:06.406330 26966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/minicluster-data/ts-1/wal/instance:
uuid: "03489ba1322e410c943615027b79d1c4"
format_stamp: "Formatted at 2026-08-06 22:45:06 on dist-test-slave-h3ft"
server_key: "e6e424020649817b3fdfc64ee0f932aa"
server_key_iv: "0f88360e094d49a2bdb4e3a05b795509"
server_key_version: "encryptionkey@0"
I20260806 22:45:06.408042 26966 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260806 22:45:06.409013 26986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:06.409247 26966 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:45:06.409372 26966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/minicluster-data/ts-1/wal
uuid: "03489ba1322e410c943615027b79d1c4"
format_stamp: "Formatted at 2026-08-06 22:45:06 on dist-test-slave-h3ft"
server_key: "e6e424020649817b3fdfc64ee0f932aa"
server_key_iv: "0f88360e094d49a2bdb4e3a05b795509"
server_key_version: "encryptionkey@0"
I20260806 22:45:06.409449 26966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:45:06.429234 26966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:45:06.432500 26966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:45:06.432679 26966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:45:06.432966 26966 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:45:06.433421 26966 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:45:06.433475 26966 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:06.433526 26966 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:45:06.433559 26966 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:06.442957 26966 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.194:35959
I20260806 22:45:06.443084 27140 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.194:35959 every 8 connection(s)
I20260806 22:45:06.443521 26966 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
I20260806 22:45:06.450790 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 26966
I20260806 22:45:06.450917 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/minicluster-data/ts-1/wal/instance
I20260806 22:45:06.451172 18615 external_mini_cluster.cc:1428] Setting key ccce0e282c63ab5115f5ec64cad31880
Aug 06 22:45:06 dist-test-slave-h3ft krb5kdc[26615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056306, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.194@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:45:06.456099 26769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:06.444497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:34059 (local address 127.18.45.254:38509)
0806 22:45:06.444576 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:06.444579 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:06.444854 (+ 275us) server_negotiation.cc:407] Connection header received
0806 22:45:06.446028 (+ 1174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:06.446030 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:06.446046 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:06.446113 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:06.446910 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:06.447624 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:06.448746 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:06.449028 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:06.449106 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:06.452003 (+ 2897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:06.452005 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:06.452005 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:06.452013 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:06.453903 (+ 1890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:06.454479 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:06.454481 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:06.454482 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:06.454570 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:06.454959 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:06.454960 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:06.454960 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:06.455148 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:06.455210 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:06.455912 (+ 702us) server_negotiation.cc:299] Negotiation successful
0806 22:45:06.456019 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 22:45:06.456254 27146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:06.444469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:38509 (local address 127.18.45.194:34059)
0806 22:45:06.444666 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:06.444712 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:06.445878 (+ 1166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:06.446203 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:06.446207 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:06.446556 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:06.446808 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:06.446813 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:06.447709 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:06.447710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:06.448636 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:06.448640 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:06.449119 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:06.449120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:06.449235 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:06.449872 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:06.449883 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:06.451886 (+ 2003us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:45:06.454011 (+ 2125us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:06.454020 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:06.454024 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:06.454356 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:06.454650 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:06.454651 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:06.454652 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:06.454871 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:06.455214 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:45:06.455222 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:45:06.455461 (+ 239us) client_negotiation.cc:769] Sending connection context
0806 22:45:06.455601 (+ 140us) client_negotiation.cc:240] Negotiation successful
0806 22:45:06.455899 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":111,"threads_started":1}
I20260806 22:45:06.456876 27143 heartbeater.cc:347] Connected to a master server at 127.18.45.254:38509
I20260806 22:45:06.457015 27143 heartbeater.cc:464] Registering TS with master...
I20260806 22:45:06.457245 27143 heartbeater.cc:511] Master 127.18.45.254:38509 requested a full tablet report, sending...
I20260806 22:45:06.457799 26684 ts_manager.cc:194] Registered new tserver with Master: 03489ba1322e410c943615027b79d1c4 (127.18.45.194:35959)
I20260806 22:45:06.458436 26684 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.45.194@KRBTEST.COM'} at 127.18.45.194:34059
WARNING: no policy specified for kudu/127.18.45.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab.
Entry for principal kudu/127.18.45.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab.
I20260806 22:45:06.487789 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.195:0
--local_ip_for_outbound_sockets=127.18.45.195
--webserver_interface=127.18.45.195
--webserver_port=0
--tserver_master_addrs=127.18.45.254:38509
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab
--principal=kudu/127.18.45.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:44455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:45:06.589962 27149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:45:06.590251 27149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:45:06.590281 27149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:45:06.591638 27149 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:45:06.591701 27149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:45:06.591732 27149 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.195
I20260806 22:45:06.593426 27149 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:44455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab
--principal=kudu/127.18.45.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.45.195
--webserver_port=0
--tserver_master_addrs=127.18.45.254:38509
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27149
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.195
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:45:06.593737 27149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:45:06.594062 27149 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:45:06.600498 27157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:06.600497 27156 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:06.600668 27149 server_base.cc:1034] running on GCE node
W20260806 22:45:06.600581 27160 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:06.601053 27149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:45:06.601401 27149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:45:06.602577 27149 hybrid_clock.cc:648] HybridClock initialized: now 1786056306602545 us; error 55 us; skew 500 ppm
Aug 06 22:45:06 dist-test-slave-h3ft krb5kdc[26615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056306, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:45:06.606148 27149 init.cc:377] Logged in from keytab as kudu/127.18.45.195@KRBTEST.COM (short username kudu)
I20260806 22:45:06.606724 27149 webserver.cc:460] Webserver started at http://127.18.45.195:35779/ using document root <none> and password file <none>
I20260806 22:45:06.606994 27149 fs_manager.cc:359] Metadata directory not provided
I20260806 22:45:06.607100 27149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:45:06.607271 27149 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:45:06.608596 27149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/minicluster-data/ts-2/data/instance:
uuid: "765b5122c5324ac88ca5ed5766f5918c"
format_stamp: "Formatted at 2026-08-06 22:45:06 on dist-test-slave-h3ft"
server_key: "3a42c3748dec3e46cc8521fb6eae561d"
server_key_iv: "775eaa16013515f85491a592274300cb"
server_key_version: "encryptionkey@0"
I20260806 22:45:06.609035 27149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/minicluster-data/ts-2/wal/instance:
uuid: "765b5122c5324ac88ca5ed5766f5918c"
format_stamp: "Formatted at 2026-08-06 22:45:06 on dist-test-slave-h3ft"
server_key: "3a42c3748dec3e46cc8521fb6eae561d"
server_key_iv: "775eaa16013515f85491a592274300cb"
server_key_version: "encryptionkey@0"
I20260806 22:45:06.610663 27149 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:45:06.611773 27169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:06.612011 27149 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:45:06.612145 27149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/minicluster-data/ts-2/wal
uuid: "765b5122c5324ac88ca5ed5766f5918c"
format_stamp: "Formatted at 2026-08-06 22:45:06 on dist-test-slave-h3ft"
server_key: "3a42c3748dec3e46cc8521fb6eae561d"
server_key_iv: "775eaa16013515f85491a592274300cb"
server_key_version: "encryptionkey@0"
I20260806 22:45:06.612241 27149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:45:06.624152 27149 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:45:06.626863 27149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:45:06.627108 27149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:45:06.627422 27149 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:45:06.627795 27149 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:45:06.627859 27149 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:06.627924 27149 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:45:06.627955 27149 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:06.635558 27149 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.195:43669
I20260806 22:45:06.635605 27337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.195:43669 every 8 connection(s)
I20260806 22:45:06.636021 27149 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
Aug 06 22:45:06 dist-test-slave-h3ft krb5kdc[26615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056306, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.195@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:45:06.646068 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 27149
I20260806 22:45:06.646215 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/minicluster-data/ts-2/wal/instance
I20260806 22:45:06.646538 18615 external_mini_cluster.cc:1428] Setting key 1068e95ea7c6146ce6af0bd144847c37
I20260806 22:45:06.647598 26769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:06.636945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.195:32849 (local address 127.18.45.254:38509)
0806 22:45:06.637047 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:06.637050 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:06.637233 (+ 183us) server_negotiation.cc:407] Connection header received
0806 22:45:06.638300 (+ 1067us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:06.638302 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:06.638315 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:06.638392 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:06.639209 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:06.639776 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:06.640878 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:06.641092 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:06.641154 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:06.643796 (+ 2642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:06.643798 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:06.643798 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:06.643807 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:06.645396 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:06.646296 (+ 900us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:06.646298 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:06.646299 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:06.646390 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:06.646738 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:06.646739 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:06.646740 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:06.646891 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:06.646979 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:06.647367 (+ 388us) server_negotiation.cc:299] Negotiation successful
0806 22:45:06.647493 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 22:45:06.647857 27340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:06.636894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:38509 (local address 127.18.45.195:32849)
0806 22:45:06.637089 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:06.637133 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:06.638194 (+ 1061us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:06.638457 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:06.638461 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:06.638772 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:06.639093 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:06.639099 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:06.639890 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:06.639891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:06.640771 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:06.640774 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:06.641192 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:06.641193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:06.641318 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:06.641950 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:06.641960 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:06.643699 (+ 1739us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:45:06.645516 (+ 1817us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:06.645528 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:06.645530 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:06.645815 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:06.646484 (+ 669us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:06.646485 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:06.646485 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:06.646656 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:06.646979 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:45:06.646989 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:45:06.647209 (+ 220us) client_negotiation.cc:769] Sending connection context
0806 22:45:06.647325 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 22:45:06.647538 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":119,"threads_started":1}
I20260806 22:45:06.648463 27338 heartbeater.cc:347] Connected to a master server at 127.18.45.254:38509
I20260806 22:45:06.648609 27338 heartbeater.cc:464] Registering TS with master...
I20260806 22:45:06.648887 27338 heartbeater.cc:511] Master 127.18.45.254:38509 requested a full tablet report, sending...
I20260806 22:45:06.649375 26684 ts_manager.cc:194] Registered new tserver with Master: 765b5122c5324ac88ca5ed5766f5918c (127.18.45.195:43669)
I20260806 22:45:06.649852 26684 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.45.195@KRBTEST.COM'} at 127.18.45.195:32849
I20260806 22:45:06.659660 18615 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/krb5kdc/test-admin.keytab.
Aug 06 22:45:06 dist-test-slave-h3ft krb5kdc[26615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056306, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:45:06.681128 18615 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260806 22:45:06.683285 18615 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/krb5kdc/krb5cc)
W20260806 22:45:06.683357 27347 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:45:06.682161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39004 (local address 127.18.45.193:41821)
0806 22:45:06.682443 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:06.682449 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:06.682464 (+ 15us) server_negotiation.cc:407] Connection header received
0806 22:45:06.682500 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:06.682502 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:06.682522 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:06.682624 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:06.683225 (+ 601us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39004: BlockingRecv error: recv got EOF from 127.0.0.1:39004 (error 108)
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":122,"threads_started":1}
Aug 06 22:45:06 dist-test-slave-h3ft krb5kdc[26615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056306, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:45:06.684593 18615 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 22:45:06 dist-test-slave-h3ft krb5kdc[26615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056306, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:45:06.685755 18615 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 22:45:06 dist-test-slave-h3ft krb5kdc[26615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056306, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.45.193@KRBTEST.COM
I20260806 22:45:06.696688 27347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:06.686483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39014 (local address 127.18.45.193:41821)
0806 22:45:06.686564 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:06.686567 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:06.686620 (+ 53us) server_negotiation.cc:407] Connection header received
0806 22:45:06.686722 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:06.686724 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:06.686743 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:06.686816 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:06.687411 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:06.688291 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:06.689244 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:06.689680 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:06.689772 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:06.692789 (+ 3017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:06.692806 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:06.692806 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:06.692817 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:06.695149 (+ 2332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:06.695627 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:06.695633 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:06.695634 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:06.695714 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:06.695952 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:06.695953 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:06.695954 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:06.696154 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:06.696382 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:06.696436 (+ 54us) server_negotiation.cc:299] Negotiation successful
0806 22:45:06.696549 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
W20260806 22:45:06.698971 27347 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:45:06.698286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39026 (local address 127.18.45.193:41821)
0806 22:45:06.698350 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:06.698352 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:06.698392 (+ 40us) server_negotiation.cc:407] Connection header received
0806 22:45:06.698461 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:06.698462 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:06.698473 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:06.698515 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:06.698896 (+ 381us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39026: BlockingRecv error: recv got EOF from 127.0.0.1:39026 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260806 22:45:06.699123 18615 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/krb5kdc/krb5cc' of type FILE
Aug 06 22:45:06 dist-test-slave-h3ft krb5kdc[26615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056306, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:45:06.700320 18615 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 22:45:06 dist-test-slave-h3ft krb5kdc[26615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056306, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:45:06.701431 18615 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 22:45:06 dist-test-slave-h3ft krb5kdc[26615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056306, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.45.193@KRBTEST.COM
I20260806 22:45:06.710137 27347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:06.702191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39030 (local address 127.18.45.193:41821)
0806 22:45:06.702246 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:06.702248 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:06.702323 (+ 75us) server_negotiation.cc:407] Connection header received
0806 22:45:06.702421 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:06.702422 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:06.702454 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:06.702526 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:06.702988 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:06.703612 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:06.704496 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:06.704733 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:06.704829 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:06.707423 (+ 2594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:06.707425 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:06.707425 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:06.707434 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:06.708834 (+ 1400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:06.709229 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:06.709230 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:06.709231 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:06.709293 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:06.709578 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:06.709580 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:06.709580 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:06.709720 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:06.709897 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:06.709970 (+ 73us) server_negotiation.cc:299] Negotiation successful
0806 22:45:06.710052 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
W20260806 22:45:06.712203 27347 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:45:06.711482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39042 (local address 127.18.45.193:41821)
0806 22:45:06.711558 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:06.711560 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:06.711643 (+ 83us) server_negotiation.cc:407] Connection header received
0806 22:45:06.711734 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:06.711735 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:06.711748 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:06.711812 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:06.712132 (+ 320us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39042: BlockingRecv error: recv got EOF from 127.0.0.1:39042 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260806 22:45:06.712363 18615 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786056275803658-18615-0/krb5kdc/krb5cc)
Aug 06 22:45:06 dist-test-slave-h3ft krb5kdc[26615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056306, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:45:06.713688 18615 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 22:45:06 dist-test-slave-h3ft krb5kdc[26615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056306, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:45:06.714886 18615 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 22:45:06 dist-test-slave-h3ft krb5kdc[26615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056306, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.45.193@KRBTEST.COM
I20260806 22:45:06.724846 27347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:06.715735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39048 (local address 127.18.45.193:41821)
0806 22:45:06.715800 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:06.715802 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:06.715896 (+ 94us) server_negotiation.cc:407] Connection header received
0806 22:45:06.716007 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:06.716008 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:06.716026 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:06.716095 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:06.716737 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:06.717558 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:06.718508 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:06.718746 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:06.718815 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:06.721719 (+ 2904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:06.721721 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:06.721721 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:06.721729 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:06.723455 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:06.723954 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:06.723956 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:06.723957 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:06.724031 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:06.724280 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:06.724280 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:06.724281 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:06.724430 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:06.724525 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:06.724645 (+ 120us) server_negotiation.cc:299] Negotiation successful
0806 22:45:06.724750 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 22:45:06.725788 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 26776
I20260806 22:45:06.743520 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 26966
I20260806 22:45:06.757874 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 27149
I20260806 22:45:06.773463 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 26634
2026-08-06T22:45:06Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2676 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:45:06 dist-test-slave-h3ft krb5kdc[27365](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:45:06 dist-test-slave-h3ft krb5kdc[27365](info): set up 2 sockets
Aug 06 22:45:06 dist-test-slave-h3ft krb5kdc[27365](info): commencing operation
krb5kdc: starting...
W20260806 22:45:08.061663 18615 mini_kdc.cc:121] Time spent starting KDC: real 1.259s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Aug 06 22:45:08 dist-test-slave-h3ft krb5kdc[27365](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056308, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:45:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:45:08Z Disabled control of system clock
WARNING: no policy specified for oryx/127.18.45.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.45.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.45.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/krb5kdc/oryx_127.18.45.254.keytab.
Entry for principal oryx/127.18.45.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/krb5kdc/oryx_127.18.45.254.keytab.
I20260806 22:45:08.202786 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:39045
--webserver_interface=127.18.45.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/krb5kdc/oryx_127.18.45.254.keytab
--principal=oryx/127.18.45.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:33219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.45.254:39045
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:45:08.299862 27383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:45:08.300108 27383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:45:08.300132 27383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:45:08.301385 27383 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:45:08.301429 27383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:45:08.301442 27383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:45:08.301455 27383 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:45:08.301467 27383 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:45:08.303143 27383 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:33219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.45.254:39045
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/krb5kdc/oryx_127.18.45.254.keytab
--principal=oryx/127.18.45.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:39045
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.45.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27383
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:45:08.303402 27383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:45:08.303627 27383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:45:08.309064 27390 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:08.309094 27395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:08.309077 27391 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:08.309543 27383 server_base.cc:1034] running on GCE node
I20260806 22:45:08.309862 27383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:45:08.310209 27383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:45:08.311395 27383 hybrid_clock.cc:648] HybridClock initialized: now 1786056308311388 us; error 56 us; skew 500 ppm
Aug 06 22:45:08 dist-test-slave-h3ft krb5kdc[27365](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056308, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.45.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:45:08.314329 27383 init.cc:377] Logged in from keytab as oryx/127.18.45.254@KRBTEST.COM (short username oryx)
I20260806 22:45:08.314792 27383 webserver.cc:460] Webserver started at http://127.18.45.254:34061/ using document root <none> and password file <none>
I20260806 22:45:08.315140 27383 fs_manager.cc:359] Metadata directory not provided
I20260806 22:45:08.315193 27383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:45:08.315376 27383 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:45:08.316609 27383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/master-0/data/instance:
uuid: "bc26ea2c399a41b3857e7d7f2a8baef6"
format_stamp: "Formatted at 2026-08-06 22:45:08 on dist-test-slave-h3ft"
server_key: "f682db2f4425d0cbda4df3ccd9a0c860"
server_key_iv: "88567c60113e59fd04390f42040e29ff"
server_key_version: "encryptionkey@0"
I20260806 22:45:08.317023 27383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/master-0/wal/instance:
uuid: "bc26ea2c399a41b3857e7d7f2a8baef6"
format_stamp: "Formatted at 2026-08-06 22:45:08 on dist-test-slave-h3ft"
server_key: "f682db2f4425d0cbda4df3ccd9a0c860"
server_key_iv: "88567c60113e59fd04390f42040e29ff"
server_key_version: "encryptionkey@0"
I20260806 22:45:08.318578 27383 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:45:08.319509 27407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:08.319741 27383 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:45:08.319860 27383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/master-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/master-0/wal
uuid: "bc26ea2c399a41b3857e7d7f2a8baef6"
format_stamp: "Formatted at 2026-08-06 22:45:08 on dist-test-slave-h3ft"
server_key: "f682db2f4425d0cbda4df3ccd9a0c860"
server_key_iv: "88567c60113e59fd04390f42040e29ff"
server_key_version: "encryptionkey@0"
I20260806 22:45:08.319953 27383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:45:08.341403 27383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:45:08.344206 27383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:45:08.344388 27383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:45:08.349249 27383 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.254:39045
I20260806 22:45:08.349301 27485 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.254:39045 every 8 connection(s)
I20260806 22:45:08.349746 27383 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
I20260806 22:45:08.351102 27486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:45:08.354019 27486 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc26ea2c399a41b3857e7d7f2a8baef6: Bootstrap starting.
I20260806 22:45:08.354722 27486 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bc26ea2c399a41b3857e7d7f2a8baef6: Neither blocks nor log segments found. Creating new log.
I20260806 22:45:08.355113 27486 log.cc:824] T 00000000000000000000000000000000 P bc26ea2c399a41b3857e7d7f2a8baef6: Log is configured to *not* fsync() on all Append() calls
I20260806 22:45:08.355924 27486 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc26ea2c399a41b3857e7d7f2a8baef6: No bootstrap required, opened a new log
I20260806 22:45:08.357717 27486 raft_consensus.cc:348] T 00000000000000000000000000000000 P bc26ea2c399a41b3857e7d7f2a8baef6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39045 } }
I20260806 22:45:08.357897 27486 raft_consensus.cc:374] T 00000000000000000000000000000000 P bc26ea2c399a41b3857e7d7f2a8baef6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:45:08.357945 27486 raft_consensus.cc:729] T 00000000000000000000000000000000 P bc26ea2c399a41b3857e7d7f2a8baef6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc26ea2c399a41b3857e7d7f2a8baef6, State: Initialized, Role: FOLLOWER
I20260806 22:45:08.358106 27486 consensus_queue.cc:260] T 00000000000000000000000000000000 P bc26ea2c399a41b3857e7d7f2a8baef6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39045 } }
I20260806 22:45:08.358191 27486 raft_consensus.cc:388] T 00000000000000000000000000000000 P bc26ea2c399a41b3857e7d7f2a8baef6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:45:08.358237 27486 raft_consensus.cc:482] T 00000000000000000000000000000000 P bc26ea2c399a41b3857e7d7f2a8baef6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:45:08.358289 27486 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bc26ea2c399a41b3857e7d7f2a8baef6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:45:08.359248 27486 raft_consensus.cc:504] T 00000000000000000000000000000000 P bc26ea2c399a41b3857e7d7f2a8baef6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39045 } }
I20260806 22:45:08.359382 27486 leader_election.cc:302] T 00000000000000000000000000000000 P bc26ea2c399a41b3857e7d7f2a8baef6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bc26ea2c399a41b3857e7d7f2a8baef6; no voters:
I20260806 22:45:08.359562 27486 leader_election.cc:288] T 00000000000000000000000000000000 P bc26ea2c399a41b3857e7d7f2a8baef6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:45:08.359607 27490 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bc26ea2c399a41b3857e7d7f2a8baef6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:45:08.359776 27490 raft_consensus.cc:686] T 00000000000000000000000000000000 P bc26ea2c399a41b3857e7d7f2a8baef6 [term 1 LEADER]: Becoming Leader. State: Replica: bc26ea2c399a41b3857e7d7f2a8baef6, State: Running, Role: LEADER
I20260806 22:45:08.359926 27490 consensus_queue.cc:237] T 00000000000000000000000000000000 P bc26ea2c399a41b3857e7d7f2a8baef6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39045 } }
I20260806 22:45:08.359989 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 27383
I20260806 22:45:08.360117 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/master-0/wal/instance
I20260806 22:45:08.360407 18615 external_mini_cluster.cc:1428] Setting key dca8f1056e0ffae1f067d9e6f38ae24a
I20260806 22:45:08.360376 27491 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc26ea2c399a41b3857e7d7f2a8baef6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39045 } } }
I20260806 22:45:08.360467 27486 sys_catalog.cc:564] T 00000000000000000000000000000000 P bc26ea2c399a41b3857e7d7f2a8baef6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:45:08.360528 27492 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc26ea2c399a41b3857e7d7f2a8baef6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc26ea2c399a41b3857e7d7f2a8baef6. Latest consensus state: current_term: 1 leader_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39045 } } }
I20260806 22:45:08.360613 27492 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc26ea2c399a41b3857e7d7f2a8baef6 [sys.catalog]: This master's current role is: LEADER
I20260806 22:45:08.360761 27491 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc26ea2c399a41b3857e7d7f2a8baef6 [sys.catalog]: This master's current role is: LEADER
I20260806 22:45:08.361819 27494 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:45:08.362481 27494 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:45:08.364576 27494 catalog_manager.cc:1347] Generated new cluster ID: f71c30cbb9674010be9e9985b316b369
I20260806 22:45:08.364634 27494 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:45:08 dist-test-slave-h3ft krb5kdc[27365](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056308, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM
I20260806 22:45:08.374060 27504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:08.361252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51410 (local address 127.18.45.254:39045)
0806 22:45:08.361433 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:08.361446 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:08.361737 (+ 291us) server_negotiation.cc:407] Connection header received
0806 22:45:08.362011 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:08.362020 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:08.362103 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:08.362244 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:08.363433 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:08.364761 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:08.365887 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:08.366530 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:08.366634 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:08.369984 (+ 3350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:08.369989 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:08.369989 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:08.370003 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:08.372150 (+ 2147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:08.372585 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:08.372588 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:08.372588 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:08.372714 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:08.372930 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:08.372931 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:08.372932 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:08.373248 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:08.373429 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:08.373535 (+ 106us) server_negotiation.cc:299] Negotiation successful
0806 22:45:08.373679 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":100,"threads_started":1}
I20260806 22:45:08.378717 27494 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:45:08.379370 27494 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:45:08.386629 27494 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bc26ea2c399a41b3857e7d7f2a8baef6: Generated new TSK 0
I20260806 22:45:08.386863 27494 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.18.45.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.45.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.45.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/krb5kdc/oryx_127.18.45.193.keytab.
Entry for principal oryx/127.18.45.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/krb5kdc/oryx_127.18.45.193.keytab.
I20260806 22:45:08.422005 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.193:0
--local_ip_for_outbound_sockets=127.18.45.193
--webserver_interface=127.18.45.193
--webserver_port=0
--tserver_master_addrs=127.18.45.254:39045
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/krb5kdc/oryx_127.18.45.193.keytab
--principal=oryx/127.18.45.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:33219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:45:08.521859 27519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:45:08.522119 27519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:45:08.522146 27519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:45:08.523381 27519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:45:08.523423 27519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:45:08.523449 27519 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.193
I20260806 22:45:08.524999 27519 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:33219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/krb5kdc/oryx_127.18.45.193.keytab
--principal=oryx/127.18.45.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.45.193
--webserver_port=0
--tserver_master_addrs=127.18.45.254:39045
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27519
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.193
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:45:08.525238 27519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:45:08.525455 27519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:45:08.531189 27527 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:08.531210 27529 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:08.531203 27526 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:08.531576 27519 server_base.cc:1034] running on GCE node
I20260806 22:45:08.531827 27519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:45:08.532122 27519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:45:08.533309 27519 hybrid_clock.cc:648] HybridClock initialized: now 1786056308533298 us; error 37 us; skew 500 ppm
Aug 06 22:45:08 dist-test-slave-h3ft krb5kdc[27365](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056308, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.45.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:45:08.536705 27519 init.cc:377] Logged in from keytab as oryx/127.18.45.193@KRBTEST.COM (short username oryx)
I20260806 22:45:08.537315 27519 webserver.cc:460] Webserver started at http://127.18.45.193:36159/ using document root <none> and password file <none>
I20260806 22:45:08.537606 27519 fs_manager.cc:359] Metadata directory not provided
I20260806 22:45:08.537679 27519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:45:08.537843 27519 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:45:08.539180 27519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-0/data/instance:
uuid: "0bc1abeccf9a48f7b80fcde36b6c6a1e"
format_stamp: "Formatted at 2026-08-06 22:45:08 on dist-test-slave-h3ft"
server_key: "1e79d8b1103916a4e1a4bd2d229d296b"
server_key_iv: "0fd99256739aeede5ac3ee8a3898742e"
server_key_version: "encryptionkey@0"
I20260806 22:45:08.539583 27519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-0/wal/instance:
uuid: "0bc1abeccf9a48f7b80fcde36b6c6a1e"
format_stamp: "Formatted at 2026-08-06 22:45:08 on dist-test-slave-h3ft"
server_key: "1e79d8b1103916a4e1a4bd2d229d296b"
server_key_iv: "0fd99256739aeede5ac3ee8a3898742e"
server_key_version: "encryptionkey@0"
I20260806 22:45:08.541121 27519 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 22:45:08.541952 27538 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:08.542148 27519 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:45:08.542260 27519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-0/wal
uuid: "0bc1abeccf9a48f7b80fcde36b6c6a1e"
format_stamp: "Formatted at 2026-08-06 22:45:08 on dist-test-slave-h3ft"
server_key: "1e79d8b1103916a4e1a4bd2d229d296b"
server_key_iv: "0fd99256739aeede5ac3ee8a3898742e"
server_key_version: "encryptionkey@0"
I20260806 22:45:08.542340 27519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:45:08.556870 27519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:45:08.559149 27519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:45:08.559326 27519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:45:08.559612 27519 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:45:08.559995 27519 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:45:08.560052 27519 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:08.560099 27519 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:45:08.560132 27519 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:08.567730 27519 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.193:37419
I20260806 22:45:08.567796 27695 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.193:37419 every 8 connection(s)
I20260806 22:45:08.568223 27519 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
I20260806 22:45:08.569528 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 27519
I20260806 22:45:08.569629 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-0/wal/instance
I20260806 22:45:08.569888 18615 external_mini_cluster.cc:1428] Setting key 3453f29b3a133c8ecb8e970708b70341
Aug 06 22:45:08 dist-test-slave-h3ft krb5kdc[27365](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056308, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.45.193@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM
I20260806 22:45:08.580657 27504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:08.569140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:51005 (local address 127.18.45.254:39045)
0806 22:45:08.569212 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:08.569215 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:08.569465 (+ 250us) server_negotiation.cc:407] Connection header received
0806 22:45:08.570275 (+ 810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:08.570278 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:08.570299 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:08.570377 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:08.571736 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:08.572574 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:08.574192 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:08.574419 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:08.574485 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:08.577440 (+ 2955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:08.577442 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:08.577443 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:08.577452 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:08.579107 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:08.579562 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:08.579564 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:08.579565 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:08.579635 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:08.579908 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:08.579909 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:08.579909 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:08.580049 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:08.580123 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:08.580441 (+ 318us) server_negotiation.cc:299] Negotiation successful
0806 22:45:08.580546 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 22:45:08.580890 27699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:08.569086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39045 (local address 127.18.45.193:51005)
0806 22:45:08.569295 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:08.569351 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:08.570150 (+ 799us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:08.570461 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:08.570466 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:08.571365 (+ 899us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:08.571620 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:08.571626 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:08.572690 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:08.572691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:08.574092 (+ 1401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:08.574096 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:08.574514 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:08.574515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:08.574687 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:08.575372 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:08.575384 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:08.577330 (+ 1946us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:45:08.579209 (+ 1879us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:08.579217 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:08.579220 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:08.579482 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:08.579709 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:08.579710 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:08.579711 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:08.579835 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:08.580123 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:45:08.580132 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:45:08.580298 (+ 166us) client_negotiation.cc:769] Sending connection context
0806 22:45:08.580406 (+ 108us) client_negotiation.cc:240] Negotiation successful
0806 22:45:08.580612 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":123,"threads_started":1}
WARNING: no policy specified for oryx/127.18.45.194@KRBTEST.COM; defaulting to no policy
I20260806 22:45:08.581471 27696 heartbeater.cc:347] Connected to a master server at 127.18.45.254:39045
I20260806 22:45:08.581611 27696 heartbeater.cc:464] Registering TS with master...
I20260806 22:45:08.581846 27696 heartbeater.cc:511] Master 127.18.45.254:39045 requested a full tablet report, sending...
I20260806 22:45:08.582516 27431 ts_manager.cc:194] Registered new tserver with Master: 0bc1abeccf9a48f7b80fcde36b6c6a1e (127.18.45.193:37419)
I20260806 22:45:08.583338 27431 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.45.193@KRBTEST.COM'} at 127.18.45.193:51005
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.45.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.45.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/krb5kdc/oryx_127.18.45.194.keytab.
Entry for principal oryx/127.18.45.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/krb5kdc/oryx_127.18.45.194.keytab.
I20260806 22:45:08.602888 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.194:0
--local_ip_for_outbound_sockets=127.18.45.194
--webserver_interface=127.18.45.194
--webserver_port=0
--tserver_master_addrs=127.18.45.254:39045
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/krb5kdc/oryx_127.18.45.194.keytab
--principal=oryx/127.18.45.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:33219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:45:08.702237 27704 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:45:08.702550 27704 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:45:08.702596 27704 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:45:08.703958 27704 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:45:08.704044 27704 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:45:08.704094 27704 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.194
I20260806 22:45:08.705928 27704 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:33219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/krb5kdc/oryx_127.18.45.194.keytab
--principal=oryx/127.18.45.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.45.194
--webserver_port=0
--tserver_master_addrs=127.18.45.254:39045
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27704
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.194
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:45:08.706264 27704 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:45:08.706552 27704 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:45:08.712895 27717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:08.712980 27713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:08.712906 27714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:08.713344 27704 server_base.cc:1034] running on GCE node
I20260806 22:45:08.713680 27704 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:45:08.714039 27704 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:45:08.715219 27704 hybrid_clock.cc:648] HybridClock initialized: now 1786056308715192 us; error 46 us; skew 500 ppm
Aug 06 22:45:08 dist-test-slave-h3ft krb5kdc[27365](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056308, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.45.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:45:08.718423 27704 init.cc:377] Logged in from keytab as oryx/127.18.45.194@KRBTEST.COM (short username oryx)
I20260806 22:45:08.718922 27704 webserver.cc:460] Webserver started at http://127.18.45.194:40585/ using document root <none> and password file <none>
I20260806 22:45:08.719236 27704 fs_manager.cc:359] Metadata directory not provided
I20260806 22:45:08.719310 27704 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:45:08.719477 27704 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:45:08.720741 27704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-1/data/instance:
uuid: "845d3ece65974e25ad6ce260ba27b693"
format_stamp: "Formatted at 2026-08-06 22:45:08 on dist-test-slave-h3ft"
server_key: "743eaa686c6789ac2bd6efc8fba723b9"
server_key_iv: "ec990f89046de94fdccf720ee6b38108"
server_key_version: "encryptionkey@0"
I20260806 22:45:08.721158 27704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-1/wal/instance:
uuid: "845d3ece65974e25ad6ce260ba27b693"
format_stamp: "Formatted at 2026-08-06 22:45:08 on dist-test-slave-h3ft"
server_key: "743eaa686c6789ac2bd6efc8fba723b9"
server_key_iv: "ec990f89046de94fdccf720ee6b38108"
server_key_version: "encryptionkey@0"
I20260806 22:45:08.722796 27704 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:45:08.723773 27724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:08.723968 27704 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:45:08.724088 27704 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-1/wal
uuid: "845d3ece65974e25ad6ce260ba27b693"
format_stamp: "Formatted at 2026-08-06 22:45:08 on dist-test-slave-h3ft"
server_key: "743eaa686c6789ac2bd6efc8fba723b9"
server_key_iv: "ec990f89046de94fdccf720ee6b38108"
server_key_version: "encryptionkey@0"
I20260806 22:45:08.724170 27704 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:45:08.741586 27704 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:45:08.744381 27704 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:45:08.744565 27704 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:45:08.744864 27704 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:45:08.745239 27704 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:45:08.745293 27704 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:08.745330 27704 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:45:08.745361 27704 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:08.752728 27704 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.194:43321
I20260806 22:45:08.752822 27886 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.194:43321 every 8 connection(s)
I20260806 22:45:08.753217 27704 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
I20260806 22:45:08.760732 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 27704
I20260806 22:45:08.760902 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-1/wal/instance
I20260806 22:45:08.761168 18615 external_mini_cluster.cc:1428] Setting key 5e148042464da38601fcc5e2d18d0993
Aug 06 22:45:08 dist-test-slave-h3ft krb5kdc[27365](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056308, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.45.194@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM
I20260806 22:45:08.766088 27504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:08.754312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:38527 (local address 127.18.45.254:39045)
0806 22:45:08.754412 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:08.754415 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:08.754637 (+ 222us) server_negotiation.cc:407] Connection header received
0806 22:45:08.755436 (+ 799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:08.755438 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:08.755454 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:08.755529 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:08.756769 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:08.757371 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:08.758484 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:08.758707 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:08.758774 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:08.762000 (+ 3226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:08.762002 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:08.762003 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:08.762012 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:08.764000 (+ 1988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:08.764575 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:08.764577 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:08.764577 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:08.764657 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:08.764972 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:08.764973 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:08.764973 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:08.765287 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:08.765414 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:08.765785 (+ 371us) server_negotiation.cc:299] Negotiation successful
0806 22:45:08.765970 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 22:45:08.766294 27891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:08.754199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39045 (local address 127.18.45.194:38527)
0806 22:45:08.754423 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:08.754481 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:08.755234 (+ 753us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:08.755652 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:08.755658 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:08.756398 (+ 740us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:08.756654 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:08.756660 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:08.757466 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:08.757467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:08.758397 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:08.758399 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:08.758794 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:08.758795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:08.758923 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:08.759653 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:08.759663 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:08.761881 (+ 2218us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:45:08.764105 (+ 2224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:08.764114 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:08.764117 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:08.764432 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:08.764747 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:08.764748 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:08.764749 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:08.764892 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:08.765411 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:45:08.765423 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:45:08.765612 (+ 189us) client_negotiation.cc:769] Sending connection context
0806 22:45:08.765745 (+ 133us) client_negotiation.cc:240] Negotiation successful
0806 22:45:08.765981 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":118,"threads_started":1}
I20260806 22:45:08.766850 27889 heartbeater.cc:347] Connected to a master server at 127.18.45.254:39045
I20260806 22:45:08.767005 27889 heartbeater.cc:464] Registering TS with master...
I20260806 22:45:08.767274 27889 heartbeater.cc:511] Master 127.18.45.254:39045 requested a full tablet report, sending...
I20260806 22:45:08.767779 27431 ts_manager.cc:194] Registered new tserver with Master: 845d3ece65974e25ad6ce260ba27b693 (127.18.45.194:43321)
I20260806 22:45:08.768216 27431 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.45.194@KRBTEST.COM'} at 127.18.45.194:38527
WARNING: no policy specified for oryx/127.18.45.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.45.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.45.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/krb5kdc/oryx_127.18.45.195.keytab.
Entry for principal oryx/127.18.45.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/krb5kdc/oryx_127.18.45.195.keytab.
I20260806 22:45:08.796614 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.195:0
--local_ip_for_outbound_sockets=127.18.45.195
--webserver_interface=127.18.45.195
--webserver_port=0
--tserver_master_addrs=127.18.45.254:39045
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/krb5kdc/oryx_127.18.45.195.keytab
--principal=oryx/127.18.45.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:33219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:45:08.901081 27897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:45:08.901363 27897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:45:08.901393 27897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:45:08.902678 27897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:45:08.902729 27897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:45:08.902760 27897 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.195
I20260806 22:45:08.904772 27897 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:33219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/krb5kdc/oryx_127.18.45.195.keytab
--principal=oryx/127.18.45.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.45.195
--webserver_port=0
--tserver_master_addrs=127.18.45.254:39045
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27897
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.195
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:45:08.905098 27897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:45:08.905369 27897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 22:45:08.911785 27897 server_base.cc:1034] running on GCE node
W20260806 22:45:08.911746 27906 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:08.911762 27910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:08.911762 27908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:08.912190 27897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:45:08.912540 27897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:45:08.913738 27897 hybrid_clock.cc:648] HybridClock initialized: now 1786056308913730 us; error 37 us; skew 500 ppm
Aug 06 22:45:08 dist-test-slave-h3ft krb5kdc[27365](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056308, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.45.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:45:08.916926 27897 init.cc:377] Logged in from keytab as oryx/127.18.45.195@KRBTEST.COM (short username oryx)
I20260806 22:45:08.917423 27897 webserver.cc:460] Webserver started at http://127.18.45.195:42419/ using document root <none> and password file <none>
I20260806 22:45:08.917696 27897 fs_manager.cc:359] Metadata directory not provided
I20260806 22:45:08.917747 27897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:45:08.917857 27897 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:45:08.919191 27897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-2/data/instance:
uuid: "06ca1eb660804c5f9d85b58de0a16cd2"
format_stamp: "Formatted at 2026-08-06 22:45:08 on dist-test-slave-h3ft"
server_key: "c55c1b23dc261ec2e7b9fea7fb5432a9"
server_key_iv: "87eae83df9dcc9c73a78e731619b180b"
server_key_version: "encryptionkey@0"
I20260806 22:45:08.919574 27897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-2/wal/instance:
uuid: "06ca1eb660804c5f9d85b58de0a16cd2"
format_stamp: "Formatted at 2026-08-06 22:45:08 on dist-test-slave-h3ft"
server_key: "c55c1b23dc261ec2e7b9fea7fb5432a9"
server_key_iv: "87eae83df9dcc9c73a78e731619b180b"
server_key_version: "encryptionkey@0"
I20260806 22:45:08.921108 27897 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:45:08.921962 27919 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:08.922144 27897 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:45:08.922230 27897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-2/wal
uuid: "06ca1eb660804c5f9d85b58de0a16cd2"
format_stamp: "Formatted at 2026-08-06 22:45:08 on dist-test-slave-h3ft"
server_key: "c55c1b23dc261ec2e7b9fea7fb5432a9"
server_key_iv: "87eae83df9dcc9c73a78e731619b180b"
server_key_version: "encryptionkey@0"
I20260806 22:45:08.922295 27897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:45:08.944049 27897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:45:08.946861 27897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:45:08.947120 27897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:45:08.947453 27897 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:45:08.947880 27897 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:45:08.947917 27897 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:08.947944 27897 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:45:08.947988 27897 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:08.955506 27897 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.195:44409
I20260806 22:45:08.955596 28073 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.195:44409 every 8 connection(s)
I20260806 22:45:08.956022 27897 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
Aug 06 22:45:08 dist-test-slave-h3ft krb5kdc[27365](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056308, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.45.195@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM
I20260806 22:45:08.964948 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 27897
I20260806 22:45:08.965096 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-2/wal/instance
I20260806 22:45:08.965385 18615 external_mini_cluster.cc:1428] Setting key ef763109f60c34e8cd93d48dd17e1883
I20260806 22:45:08.968003 27504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:08.957061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.195:58663 (local address 127.18.45.254:39045)
0806 22:45:08.957145 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:08.957148 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:08.957404 (+ 256us) server_negotiation.cc:407] Connection header received
0806 22:45:08.958032 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:08.958033 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:08.958052 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:08.958124 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:08.959409 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:08.959961 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:08.961187 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:08.961514 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:08.961582 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:08.964379 (+ 2797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:08.964382 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:08.964382 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:08.964391 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:08.966403 (+ 2012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:08.966978 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:08.966979 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:08.966979 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:08.967098 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:08.967328 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:08.967328 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:08.967329 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:08.967467 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:08.967538 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:08.967825 (+ 287us) server_negotiation.cc:299] Negotiation successful
0806 22:45:08.967905 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 22:45:08.968289 28078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:08.957001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39045 (local address 127.18.45.195:58663)
0806 22:45:08.957246 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:08.957300 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:08.957927 (+ 627us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:08.958216 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:08.958220 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:08.958921 (+ 701us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:08.959306 (+ 385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:08.959313 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:08.960054 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:08.960055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:08.961074 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:08.961077 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:08.961607 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:08.961608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:08.961737 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:08.962499 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:08.962510 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:08.964263 (+ 1753us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:45:08.966583 (+ 2320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:08.966592 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:08.966594 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:08.966884 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:08.967148 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:08.967149 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:08.967149 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:08.967272 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:08.967531 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:45:08.967536 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:45:08.967691 (+ 155us) client_negotiation.cc:769] Sending connection context
0806 22:45:08.967796 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 22:45:08.967960 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":110,"threads_started":1}
I20260806 22:45:08.968894 28074 heartbeater.cc:347] Connected to a master server at 127.18.45.254:39045
I20260806 22:45:08.969050 28074 heartbeater.cc:464] Registering TS with master...
I20260806 22:45:08.969303 28074 heartbeater.cc:511] Master 127.18.45.254:39045 requested a full tablet report, sending...
I20260806 22:45:08.969801 27431 ts_manager.cc:194] Registered new tserver with Master: 06ca1eb660804c5f9d85b58de0a16cd2 (127.18.45.195:44409)
I20260806 22:45:08.970222 27431 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.45.195@KRBTEST.COM'} at 127.18.45.195:58663
I20260806 22:45:08.977682 18615 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.18.45.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.45.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.45.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/krb5kdc/oryx_127.18.45.253.keytab.
Entry for principal oryx/127.18.45.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/krb5kdc/oryx_127.18.45.253.keytab.
I20260806 22:45:09.012130 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.253:40509
--webserver_interface=127.18.45.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/krb5kdc/oryx_127.18.45.253.keytab
--principal=oryx/127.18.45.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:33219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.45.254:39045,127.18.45.253:40509
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:45:09.116802 28083 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:45:09.117085 28083 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:45:09.117111 28083 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:45:09.118316 28083 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:45:09.118359 28083 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:45:09.118374 28083 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:45:09.118386 28083 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:45:09.118399 28083 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:45:09.120113 28083 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:33219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.18.45.254:39045,127.18.45.253:40509
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/krb5kdc/oryx_127.18.45.253.keytab
--principal=oryx/127.18.45.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.253:40509
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.45.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28083
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
W20260806 22:45:09.120254 28083 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.18.45.254:39045,127.18.45.253:40509'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260806 22:45:09.120407 28083 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:45:09.120638 28083 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:45:09.126628 28092 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:09.126585 28090 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:09.126953 28083 server_base.cc:1034] running on GCE node
W20260806 22:45:09.126598 28089 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:09.127334 28083 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:45:09.127781 28083 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:45:09.129025 28083 hybrid_clock.cc:648] HybridClock initialized: now 1786056309128999 us; error 46 us; skew 500 ppm
Aug 06 22:45:09 dist-test-slave-h3ft krb5kdc[27365](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056309, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.45.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:45:09.132207 28083 init.cc:377] Logged in from keytab as oryx/127.18.45.253@KRBTEST.COM (short username oryx)
I20260806 22:45:09.132804 28083 webserver.cc:460] Webserver started at http://127.18.45.253:43995/ using document root <none> and password file <none>
I20260806 22:45:09.133080 28083 fs_manager.cc:359] Metadata directory not provided
I20260806 22:45:09.133131 28083 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:45:09.133277 28083 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:45:09.134574 28083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/master-1/data/instance:
uuid: "8f1bacf8b0b7450f8cff03a77e4c0890"
format_stamp: "Formatted at 2026-08-06 22:45:09 on dist-test-slave-h3ft"
server_key: "52e971c5a5eaf3c76a4ba2b6edc2da1b"
server_key_iv: "d28c02bd5c7f22470c2d6428169caaae"
server_key_version: "encryptionkey@0"
I20260806 22:45:09.135052 28083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/master-1/wal/instance:
uuid: "8f1bacf8b0b7450f8cff03a77e4c0890"
format_stamp: "Formatted at 2026-08-06 22:45:09 on dist-test-slave-h3ft"
server_key: "52e971c5a5eaf3c76a4ba2b6edc2da1b"
server_key_iv: "d28c02bd5c7f22470c2d6428169caaae"
server_key_version: "encryptionkey@0"
I20260806 22:45:09.136970 28083 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.001s
I20260806 22:45:09.137909 28101 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:09.138121 28083 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:45:09.138231 28083 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/master-1/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/master-1/wal
uuid: "8f1bacf8b0b7450f8cff03a77e4c0890"
format_stamp: "Formatted at 2026-08-06 22:45:09 on dist-test-slave-h3ft"
server_key: "52e971c5a5eaf3c76a4ba2b6edc2da1b"
server_key_iv: "d28c02bd5c7f22470c2d6428169caaae"
server_key_version: "encryptionkey@0"
I20260806 22:45:09.138293 28083 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:45:09.157491 28083 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:45:09.160221 28083 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:45:09.160405 28083 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:45:09.165936 28083 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.253:40509
I20260806 22:45:09.165977 28173 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.253:40509 every 8 connection(s)
I20260806 22:45:09.166467 28083 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/master-1/data/info.pb
I20260806 22:45:09.167919 28174 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:45:09.170261 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 28083
I20260806 22:45:09.170113 28174 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39045 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:45:09 dist-test-slave-h3ft krb5kdc[27365](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056309, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.45.253@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM
Aug 06 22:45:09 dist-test-slave-h3ft krb5kdc[27365](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056308, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.45.253@KRBTEST.COM
I20260806 22:45:09.182027 27504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:09.171267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55994 (local address 127.18.45.254:39045)
0806 22:45:09.171368 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:09.171372 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:09.171647 (+ 275us) server_negotiation.cc:407] Connection header received
0806 22:45:09.172314 (+ 667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:09.172316 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:09.172336 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:09.172432 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:09.173131 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.173707 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.174729 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.174951 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.175081 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:09.177933 (+ 2852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:09.177935 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:09.177935 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:09.177945 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:09.179882 (+ 1937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:09.180433 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:09.180434 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:09.180435 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:09.180499 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:09.180839 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:09.180841 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:09.180841 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:09.181158 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:09.181219 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:09.181713 (+ 494us) server_negotiation.cc:299] Negotiation successful
0806 22:45:09.181837 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 22:45:09.182246 28180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:09.171258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55716 (local address 127.18.45.253:40509)
0806 22:45:09.171607 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:09.171611 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:09.171640 (+ 29us) server_negotiation.cc:407] Connection header received
0806 22:45:09.172190 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:09.172205 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:09.172233 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:09.172352 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:09.173092 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.174024 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.174792 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.175193 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.175282 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:09.178349 (+ 3067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:09.178360 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:09.178361 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:09.178387 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:09.180606 (+ 2219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:09.181092 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:09.181096 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:09.181096 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:09.181169 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:09.181422 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:09.181423 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:09.181423 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:09.181628 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:09.181807 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:09.181930 (+ 123us) server_negotiation.cc:299] Negotiation successful
0806 22:45:09.182017 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":143,"threads_started":1}
I20260806 22:45:09.182528 28179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:09.171213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39045 (local address 127.0.0.1:55994)
0806 22:45:09.171490 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:09.171531 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:09.172179 (+ 648us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:09.172466 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:09.172469 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:09.172799 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:09.173017 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:09.173021 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.173846 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.173848 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:09.174613 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:09.174615 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.175092 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.175093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:09.175224 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:09.175870 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:09.175883 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:09.177754 (+ 1871us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:45:09.179970 (+ 2216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:09.179976 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:09.179979 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:09.180347 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:09.180567 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:09.180568 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:09.180568 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:09.180749 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:09.181232 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:45:09.181240 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:45:09.181460 (+ 220us) client_negotiation.cc:769] Sending connection context
0806 22:45:09.181603 (+ 143us) client_negotiation.cc:240] Negotiation successful
0806 22:45:09.181823 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":164,"threads_started":1}
I20260806 22:45:09.183754 28174 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.45.253" port: 40509 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:45:09 dist-test-slave-h3ft krb5kdc[27365](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056309, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.45.253@KRBTEST.COM for oryx/127.18.45.253@KRBTEST.COM
I20260806 22:45:09.191813 28179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:09.184123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.253:40509 (local address 127.0.0.1:55730)
0806 22:45:09.184164 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:09.184176 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:09.184221 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:09.184417 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:09.184419 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:09.184570 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:09.184617 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:09.184618 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.185403 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.185404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:09.185914 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:09.185916 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.186321 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.186322 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:09.186377 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:09.186918 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:09.186943 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:09.188578 (+ 1635us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:45:09.190534 (+ 1956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:09.190535 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:09.190536 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:09.190702 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:09.191072 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:09.191073 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:09.191078 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:09.191147 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:09.191469 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:45:09.191470 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:45:09.191516 (+ 46us) client_negotiation.cc:769] Sending connection context
0806 22:45:09.191575 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 22:45:09.191684 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260806 22:45:09.191818 28180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:09.184164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55730 (local address 127.18.45.253:40509)
0806 22:45:09.184225 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:09.184227 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:09.184237 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:45:09.184297 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:09.184297 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:09.184311 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:09.184374 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:09.184694 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.185288 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.186000 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.186236 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.186302 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:09.188675 (+ 2373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:09.188677 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:09.188677 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:09.188685 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:09.190436 (+ 1751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:09.190805 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:09.190806 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:09.190807 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:09.190921 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:09.191233 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:09.191234 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:09.191235 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:09.191392 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:09.191471 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:09.191602 (+ 131us) server_negotiation.cc:299] Negotiation successful
0806 22:45:09.191736 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":57,"server-negotiator.queue_time_us":38}
I20260806 22:45:09.193773 28174 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890: Bootstrap starting.
I20260806 22:45:09.194943 28174 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890: Neither blocks nor log segments found. Creating new log.
I20260806 22:45:09.195367 28174 log.cc:824] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890: Log is configured to *not* fsync() on all Append() calls
I20260806 22:45:09.196427 28174 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890: No bootstrap required, opened a new log
I20260806 22:45:09.198237 28174 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39045 } } peers { permanent_uuid: "8f1bacf8b0b7450f8cff03a77e4c0890" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 40509 } }
I20260806 22:45:09.198400 28174 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:45:09.198426 28174 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f1bacf8b0b7450f8cff03a77e4c0890, State: Initialized, Role: FOLLOWER
I20260806 22:45:09.198544 28174 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39045 } } peers { permanent_uuid: "8f1bacf8b0b7450f8cff03a77e4c0890" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 40509 } }
I20260806 22:45:09.198915 28174 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:45:09.198879 28184 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39045 } } peers { permanent_uuid: "8f1bacf8b0b7450f8cff03a77e4c0890" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 40509 } } }
I20260806 22:45:09.199132 28184 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 22:45:09.201222 28200 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:45:09.201306 28200 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:45:09.203820 28083 master_runner.cc:428] Detected that this master 8f1bacf8b0b7450f8cff03a77e4c0890 is joining an existing cluster
I20260806 22:45:09.203959 28083 master_runner.cc:432] Initiating AddMaster RPC to add 127.18.45.253:40509
I20260806 22:45:09.213833 27504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:09.205426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56008 (local address 127.18.45.254:39045)
0806 22:45:09.205493 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:09.205496 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:09.205671 (+ 175us) server_negotiation.cc:407] Connection header received
0806 22:45:09.205790 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:09.205791 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:09.205807 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:09.205842 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:09.206478 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.207136 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.208354 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.208571 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.208605 (+ 34us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:09.210491 (+ 1886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:09.210492 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:09.210493 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:09.210502 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:09.212344 (+ 1842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:09.212813 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:09.212815 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:09.212816 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:09.212893 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:09.213128 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:09.213130 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:09.213130 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:09.213311 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:09.213398 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:09.213607 (+ 209us) server_negotiation.cc:299] Negotiation successful
0806 22:45:09.213716 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 22:45:09.213855 28212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:09.205355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39045 (local address 127.0.0.1:56008)
0806 22:45:09.205570 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:09.205598 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:09.205704 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:09.205935 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:09.205935 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:09.206242 (+ 307us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:09.206379 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:09.206382 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.207244 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.207245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:09.208235 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:09.208238 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.208698 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.208699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:09.208785 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:09.209643 (+ 858us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:09.209649 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:09.210343 (+ 694us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:45:09.212456 (+ 2113us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:09.212457 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:09.212457 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:09.212730 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:09.212984 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:09.212985 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:09.212985 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:09.213050 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:09.213433 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:45:09.213434 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:45:09.213492 (+ 58us) client_negotiation.cc:769] Sending connection context
0806 22:45:09.213566 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 22:45:09.213703 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"mutex_wait_us":2,"thread_start_us":145,"threads_started":1}
I20260806 22:45:09.214079 28180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:09.205631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55738 (local address 127.18.45.253:40509)
0806 22:45:09.205675 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:09.205677 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:09.205721 (+ 44us) server_negotiation.cc:407] Connection header received
0806 22:45:09.205789 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:09.205789 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:09.205802 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:09.205861 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:09.206490 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.207372 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.208356 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.208662 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.208742 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:09.210443 (+ 1701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:09.210445 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:09.210446 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:09.210455 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:09.212849 (+ 2394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:09.213270 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:09.213271 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:09.213271 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:09.213328 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:09.213533 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:09.213534 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:09.213534 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:09.213668 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:09.213734 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:09.213907 (+ 173us) server_negotiation.cc:299] Negotiation successful
0806 22:45:09.213997 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 22:45:09.214359 28208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:09.205387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.253:40509 (local address 127.0.0.1:55738)
0806 22:45:09.205643 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:09.205653 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:09.205737 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:09.205921 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:09.205921 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:09.206255 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:09.206389 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:09.206392 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.207462 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.207463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:09.208261 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:09.208263 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.208746 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.208747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:09.208807 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:09.209562 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:09.209567 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:09.210343 (+ 776us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:45:09.212945 (+ 2602us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:09.212946 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:09.212946 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:09.213194 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:09.213406 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:09.213406 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:09.213407 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:09.213467 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:09.213757 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:45:09.213758 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:45:09.213792 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 22:45:09.214111 (+ 319us) client_negotiation.cc:240] Negotiation successful
0806 22:45:09.214261 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"mutex_wait_us":5,"thread_start_us":97,"threads_started":1}
Aug 06 22:45:09 dist-test-slave-h3ft krb5kdc[27365](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056308, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.45.254@KRBTEST.COM for oryx/127.18.45.253@KRBTEST.COM
I20260806 22:45:09.225562 28180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:09.217619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55750 (local address 127.18.45.253:40509)
0806 22:45:09.217674 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:09.217676 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:09.217823 (+ 147us) server_negotiation.cc:407] Connection header received
0806 22:45:09.217935 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:09.217936 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:09.217949 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:09.217990 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:09.218521 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.219096 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.219928 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.220142 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.220218 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:09.222529 (+ 2311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:09.222531 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:09.222531 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:09.222540 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:09.224177 (+ 1637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:09.224612 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:09.224613 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:09.224614 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:09.224688 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:09.224894 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:09.224895 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:09.224895 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:09.225033 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:09.225107 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:09.225333 (+ 226us) server_negotiation.cc:299] Negotiation successful
0806 22:45:09.225449 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 22:45:09.225567 28214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:09.217516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.253:40509 (local address 127.0.0.1:55750)
0806 22:45:09.217689 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:09.217753 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:09.217856 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:09.218011 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:09.218015 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:09.218363 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:09.218436 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:09.218439 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.219177 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.219178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:09.219840 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:09.219842 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.220217 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.220218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:09.220314 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:09.220852 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:09.220857 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:09.222429 (+ 1572us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:45:09.224272 (+ 1843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:09.224283 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:09.224286 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:09.224529 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:09.224761 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:09.224762 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:09.224763 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:09.224830 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:09.225107 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:45:09.225117 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:45:09.225187 (+ 70us) client_negotiation.cc:769] Sending connection context
0806 22:45:09.225281 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 22:45:09.225439 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":81,"threads_started":1}
I20260806 22:45:09.226307 27431 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.18.45.253:40509: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "8f1bacf8b0b7450f8cff03a77e4c0890"
member_type: NON_VOTER
last_known_addr {
host: "127.18.45.253"
port: 40509
}
attrs {
promote: true
}
}
dest_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6"
cas_config_opid_index: -1
I20260806 22:45:09.226781 27431 consensus_queue.cc:237] T 00000000000000000000000000000000 P bc26ea2c399a41b3857e7d7f2a8baef6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39045 } } peers { permanent_uuid: "8f1bacf8b0b7450f8cff03a77e4c0890" member_type: NON_VOTER last_known_addr { host: "127.18.45.253" port: 40509 } attrs { promote: true } }
I20260806 22:45:09.227490 28218 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bc26ea2c399a41b3857e7d7f2a8baef6 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 8f1bacf8b0b7450f8cff03a77e4c0890 (127.18.45.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39045 } } peers { permanent_uuid: "8f1bacf8b0b7450f8cff03a77e4c0890" member_type: NON_VOTER last_known_addr { host: "127.18.45.253" port: 40509 } attrs { promote: true } } }
I20260806 22:45:09.228017 28141 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:45:09.228940 28218 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.18.45.253:40509
I20260806 22:45:09.229059 28216 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc26ea2c399a41b3857e7d7f2a8baef6 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39045 } } peers { permanent_uuid: "8f1bacf8b0b7450f8cff03a77e4c0890" member_type: NON_VOTER last_known_addr { host: "127.18.45.253" port: 40509 } attrs { promote: true } } }
I20260806 22:45:09.229197 28216 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc26ea2c399a41b3857e7d7f2a8baef6 [sys.catalog]: This master's current role is: LEADER
I20260806 22:45:09.229264 28141 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890 [term 1 FOLLOWER]: Refusing update from remote peer bc26ea2c399a41b3857e7d7f2a8baef6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260806 22:45:09.229696 28216 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bc26ea2c399a41b3857e7d7f2a8baef6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f1bacf8b0b7450f8cff03a77e4c0890" member_type: NON_VOTER last_known_addr { host: "127.18.45.253" port: 40509 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:45:09.229785 28083 master.cc:537] Master@127.18.45.253:40509 shutting down...
W20260806 22:45:09.231112 27409 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260806 22:45:09.232915 28083 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:45:09.233026 28083 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:45:09.233069 28083 tablet_replica.cc:330] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890: stopping tablet replica
I20260806 22:45:09.239530 28214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:09.239369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.253:40509 (local address 127.0.0.1:55760)
0806 22:45:09.239437 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:09.239483 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.253:40509: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260806 22:45:09.239655 27409 consensus_peers.cc:487] T 00000000000000000000000000000000 P bc26ea2c399a41b3857e7d7f2a8baef6 -> Peer 8f1bacf8b0b7450f8cff03a77e4c0890 (127.18.45.253:40509): Couldn't send request to peer 8f1bacf8b0b7450f8cff03a77e4c0890. Status: Network error: Client connection negotiation failed: client connection to 127.18.45.253:40509: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 22:45:09.247291 28083 master.cc:552] Master@127.18.45.253:40509 shutdown complete.
I20260806 22:45:09.247380 28083 master_runner.cc:305] Clearing existing system tablet
I20260806 22:45:09.247599 28083 fs_manager.cc:359] Metadata directory not provided
I20260806 22:45:09.247665 28083 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:45:09.248594 28083 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:45:09.249215 28223 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:09.249377 28083 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:45:09.249451 28083 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:45:09.249477 28083 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/master-1/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/master-1/wal
uuid: "8f1bacf8b0b7450f8cff03a77e4c0890"
format_stamp: "Formatted at 2026-08-06 22:45:09 on dist-test-slave-h3ft"
server_key: "52e971c5a5eaf3c76a4ba2b6edc2da1b"
server_key_iv: "d28c02bd5c7f22470c2d6428169caaae"
server_key_version: "encryptionkey@0"
I20260806 22:45:09.250582 28083 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:45:09.252578 28083 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 22:45:09.252665 28083 log.cc:1196] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890: Deleting WAL directory at /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260806 22:45:09.252914 28083 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890: Deleting consensus metadata
I20260806 22:45:09.253057 28083 master_runner.cc:315] Copying system tablet from 127.18.45.254:39045
I20260806 22:45:09.253846 28083 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890: tablet copy: Beginning tablet copy session from remote peer at address 127.18.45.254:39045
I20260806 22:45:09.261685 27504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:09.254600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56016 (local address 127.18.45.254:39045)
0806 22:45:09.254688 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:09.254692 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:09.254823 (+ 131us) server_negotiation.cc:407] Connection header received
0806 22:45:09.254991 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:09.254993 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:09.255049 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:09.255127 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:09.255558 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.256186 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.257051 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.257274 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.257388 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:09.258622 (+ 1234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:09.258624 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:09.258625 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:09.258633 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:09.260285 (+ 1652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:09.260684 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:09.260685 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:09.260686 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:09.260758 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:09.261030 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:09.261031 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:09.261032 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:09.261181 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:09.261269 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:09.261470 (+ 201us) server_negotiation.cc:299] Negotiation successful
0806 22:45:09.261565 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 22:45:09.261696 28231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:09.254522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39045 (local address 127.0.0.1:56016)
0806 22:45:09.254728 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:09.254742 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:09.254844 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:09.255134 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:09.255136 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:09.255387 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:09.255466 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:09.255469 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.256281 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.256282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:09.256955 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:09.256957 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.257387 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.257388 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:09.257472 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:09.258134 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:09.258140 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:09.258520 (+ 380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:45:09.260392 (+ 1872us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:09.260394 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:09.260395 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:09.260604 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:09.260830 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:09.260831 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:09.260831 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:09.260953 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:09.261265 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:45:09.261266 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:45:09.261325 (+ 59us) client_negotiation.cc:769] Sending connection context
0806 22:45:09.261426 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 22:45:09.261563 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":111,"threads_started":1}
I20260806 22:45:09.262554 27461 tablet_copy_service.cc:140] P bc26ea2c399a41b3857e7d7f2a8baef6: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 8f1bacf8b0b7450f8cff03a77e4c0890 ({username='oryx', principal='oryx/127.18.45.253@KRBTEST.COM'} at 127.0.0.1:56016)
I20260806 22:45:09.262693 27461 tablet_copy_service.cc:161] P bc26ea2c399a41b3857e7d7f2a8baef6: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 8f1bacf8b0b7450f8cff03a77e4c0890 at {username='oryx', principal='oryx/127.18.45.253@KRBTEST.COM'} at 127.0.0.1:56016: session id = 8f1bacf8b0b7450f8cff03a77e4c0890-00000000000000000000000000000000
I20260806 22:45:09.263789 27461 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P bc26ea2c399a41b3857e7d7f2a8baef6: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 22:45:09.264632 28083 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 22:45:09.266324 28083 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890: tablet copy: Starting download of 0 data blocks...
I20260806 22:45:09.266530 28083 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890: tablet copy: Starting download of 1 WAL segments...
I20260806 22:45:09.267700 28083 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 22:45:09.268724 27461 tablet_copy_service.cc:342] P bc26ea2c399a41b3857e7d7f2a8baef6: Request end of tablet copy session 8f1bacf8b0b7450f8cff03a77e4c0890-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.18.45.253@KRBTEST.COM'} at 127.0.0.1:56016
I20260806 22:45:09.268786 27461 tablet_copy_service.cc:433] P bc26ea2c399a41b3857e7d7f2a8baef6: ending tablet copy session 8f1bacf8b0b7450f8cff03a77e4c0890-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 8f1bacf8b0b7450f8cff03a77e4c0890
I20260806 22:45:09.272523 28083 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:45:09.273880 28232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:09.273990 28233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:09.276106 28238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:09.276178 28083 server_base.cc:1034] running on GCE node
I20260806 22:45:09.276383 28083 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:45:09.276605 28083 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:45:09.277760 28083 hybrid_clock.cc:648] HybridClock initialized: now 1786056309277742 us; error 39 us; skew 500 ppm
Aug 06 22:45:09 dist-test-slave-h3ft krb5kdc[27365](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056309, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.45.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:45:09.279627 28083 init.cc:377] Logged in from keytab as oryx/127.18.45.253@KRBTEST.COM (short username oryx)
I20260806 22:45:09.280038 28083 webserver.cc:460] Webserver started at http://127.18.45.253:44397/ using document root <none> and password file <none>
I20260806 22:45:09.280186 28083 fs_manager.cc:359] Metadata directory not provided
I20260806 22:45:09.280238 28083 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:45:09.281088 28083 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:45:09.281739 28253 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:09.281909 28083 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:09.281977 28083 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/master-1/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/master-1/wal
uuid: "8f1bacf8b0b7450f8cff03a77e4c0890"
format_stamp: "Formatted at 2026-08-06 22:45:09 on dist-test-slave-h3ft"
server_key: "52e971c5a5eaf3c76a4ba2b6edc2da1b"
server_key_iv: "d28c02bd5c7f22470c2d6428169caaae"
server_key_version: "encryptionkey@0"
I20260806 22:45:09.282052 28083 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:45:09.292083 28083 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:45:09.294234 28083 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:45:09.298677 28083 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.253:40509
I20260806 22:45:09.298724 28323 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.253:40509 every 8 connection(s)
I20260806 22:45:09.299134 28083 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/master-1/data/info.pb
I20260806 22:45:09.300390 28324 sys_catalog.cc:263] Verifying existing consensus state
I20260806 22:45:09.300931 28324 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890: Bootstrap starting.
I20260806 22:45:09.304149 28324 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:45:09.304406 28324 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890: Bootstrap complete.
I20260806 22:45:09.304659 28324 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39045 } } peers { permanent_uuid: "8f1bacf8b0b7450f8cff03a77e4c0890" member_type: NON_VOTER last_known_addr { host: "127.18.45.253" port: 40509 } attrs { promote: true } }
I20260806 22:45:09.304751 28324 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 8f1bacf8b0b7450f8cff03a77e4c0890, State: Initialized, Role: LEARNER
I20260806 22:45:09.304783 28324 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39045 } } peers { permanent_uuid: "8f1bacf8b0b7450f8cff03a77e4c0890" member_type: NON_VOTER last_known_addr { host: "127.18.45.253" port: 40509 } attrs { promote: true } }
I20260806 22:45:09.305066 28324 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:45:09.305047 28327 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39045 } } peers { permanent_uuid: "8f1bacf8b0b7450f8cff03a77e4c0890" member_type: NON_VOTER last_known_addr { host: "127.18.45.253" port: 40509 } attrs { promote: true } } }
I20260806 22:45:09.305151 28327 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890 [sys.catalog]: This master's current role is: LEARNER
I20260806 22:45:09.306874 28339 catalog_manager.cc:1259] Loaded cluster ID: f71c30cbb9674010be9e9985b316b369
I20260806 22:45:09.306944 28339 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890: loading cluster ID for follower catalog manager: success
I20260806 22:45:09.308112 28339 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890: acquiring CA information for follower catalog manager: success
I20260806 22:45:09.308451 28339 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 22:45:09.343652 28342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:09.335969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55774 (local address 127.18.45.253:40509)
0806 22:45:09.336194 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:09.336196 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:09.336206 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:45:09.336238 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:09.336239 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:09.336250 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:09.336331 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:09.337038 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.337825 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.338628 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.338855 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.338938 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:09.340473 (+ 1535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:09.340475 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:09.340475 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:09.340483 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:09.342183 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:09.342591 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:09.342593 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:09.342593 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:09.342684 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:09.342895 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:09.342896 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:09.342896 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:09.343102 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:09.343238 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:09.343378 (+ 140us) server_negotiation.cc:299] Negotiation successful
0806 22:45:09.343487 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":142,"threads_started":1}
WARNING: no policy specified for oryx/127.18.45.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.45.252@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.45.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/krb5kdc/oryx_127.18.45.252.keytab.
Entry for principal oryx/127.18.45.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/krb5kdc/oryx_127.18.45.252.keytab.
I20260806 22:45:09.381696 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.252:40869
--webserver_interface=127.18.45.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/krb5kdc/oryx_127.18.45.252.keytab
--principal=oryx/127.18.45.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:33219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.45.254:39045,127.18.45.253:40509,127.18.45.252:40869
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:45:09.485931 28349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:45:09.486231 28349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:45:09.486275 28349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:45:09.487663 28349 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:45:09.487733 28349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:45:09.487768 28349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:45:09.487813 28349 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:45:09.487843 28349 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:45:09.489578 28349 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:33219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.18.45.254:39045,127.18.45.253:40509,127.18.45.252:40869
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/krb5kdc/oryx_127.18.45.252.keytab
--principal=oryx/127.18.45.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.252:40869
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.45.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28349
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:45:09.489931 28349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:45:09.490213 28349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:45:09.496104 28357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:09.496177 28359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:09.496177 28361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:09.496413 28349 server_base.cc:1034] running on GCE node
I20260806 22:45:09.496744 28349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:45:09.497112 28349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:45:09.498304 28349 hybrid_clock.cc:648] HybridClock initialized: now 1786056309498289 us; error 45 us; skew 500 ppm
Aug 06 22:45:09 dist-test-slave-h3ft krb5kdc[27365](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056309, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.45.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:45:09.501745 28349 init.cc:377] Logged in from keytab as oryx/127.18.45.252@KRBTEST.COM (short username oryx)
I20260806 22:45:09.502390 28349 webserver.cc:460] Webserver started at http://127.18.45.252:46817/ using document root <none> and password file <none>
I20260806 22:45:09.502676 28349 fs_manager.cc:359] Metadata directory not provided
I20260806 22:45:09.502750 28349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:45:09.502915 28349 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:45:09.504431 28349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/master-2/data/instance:
uuid: "69b1737c4ee248689c79e5644bc1274d"
format_stamp: "Formatted at 2026-08-06 22:45:09 on dist-test-slave-h3ft"
server_key: "b1621c0123e4751e846ec93a491e176d"
server_key_iv: "2b96ab35c6718477644b1a6c67a313b6"
server_key_version: "encryptionkey@0"
I20260806 22:45:09.504880 28349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/master-2/wal/instance:
uuid: "69b1737c4ee248689c79e5644bc1274d"
format_stamp: "Formatted at 2026-08-06 22:45:09 on dist-test-slave-h3ft"
server_key: "b1621c0123e4751e846ec93a491e176d"
server_key_iv: "2b96ab35c6718477644b1a6c67a313b6"
server_key_version: "encryptionkey@0"
I20260806 22:45:09.507264 28349 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 22:45:09.508319 28369 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:09.508527 28349 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:45:09.508646 28349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/master-2/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/master-2/wal
uuid: "69b1737c4ee248689c79e5644bc1274d"
format_stamp: "Formatted at 2026-08-06 22:45:09 on dist-test-slave-h3ft"
server_key: "b1621c0123e4751e846ec93a491e176d"
server_key_iv: "2b96ab35c6718477644b1a6c67a313b6"
server_key_version: "encryptionkey@0"
I20260806 22:45:09.508740 28349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:45:09.521694 28349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:45:09.524916 28349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:45:09.525161 28349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:45:09.531595 28349 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.252:40869
I20260806 22:45:09.531664 28450 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.252:40869 every 8 connection(s)
I20260806 22:45:09.532128 28349 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/master-2/data/info.pb
I20260806 22:45:09.533500 28451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:45:09.535681 28451 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39045 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:45:09.539611 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 28349
Aug 06 22:45:09 dist-test-slave-h3ft krb5kdc[27365](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056309, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.45.252@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM
Aug 06 22:45:09 dist-test-slave-h3ft krb5kdc[27365](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056308, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.45.252@KRBTEST.COM
I20260806 22:45:09.548077 27504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:09.536760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56028 (local address 127.18.45.254:39045)
0806 22:45:09.536841 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:09.536844 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:09.537144 (+ 300us) server_negotiation.cc:407] Connection header received
0806 22:45:09.537782 (+ 638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:09.537783 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:09.537800 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:09.537866 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:09.538719 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.539342 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.540762 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.540990 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.541073 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:09.543962 (+ 2889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:09.543964 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:09.543964 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:09.543974 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:09.545968 (+ 1994us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:09.546673 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:09.546675 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:09.546675 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:09.546763 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:09.547180 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:09.547181 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:09.547181 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:09.547345 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:09.547450 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:09.547837 (+ 387us) server_negotiation.cc:299] Negotiation successful
0806 22:45:09.547953 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 22:45:09.548390 28453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:09.536603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39045 (local address 127.0.0.1:56028)
0806 22:45:09.536920 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:09.537014 (+ 94us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:09.537667 (+ 653us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:09.537995 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:09.537999 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:09.538329 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:09.538603 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:09.538607 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.539448 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.539449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:09.540671 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:09.540674 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.541123 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.541124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:09.541265 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:09.541904 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:09.541914 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:09.543838 (+ 1924us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:45:09.546098 (+ 2260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:09.546111 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:09.546114 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:09.546562 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:09.546868 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:09.546869 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:09.546870 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:09.547079 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:09.547438 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:45:09.547448 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:45:09.547642 (+ 194us) client_negotiation.cc:769] Sending connection context
0806 22:45:09.547776 (+ 134us) client_negotiation.cc:240] Negotiation successful
0806 22:45:09.548018 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":122,"threads_started":1}
I20260806 22:45:09.549203 28451 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.45.253" port: 40509 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:45:09.550033 28456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:09.540354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52726 (local address 127.18.45.252:40869)
0806 22:45:09.540598 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:09.540602 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:09.540620 (+ 18us) server_negotiation.cc:407] Connection header received
0806 22:45:09.540639 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:09.540641 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:09.540671 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:09.540774 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:09.541487 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.542238 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.543246 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.543678 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.543747 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:09.546661 (+ 2914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:09.546671 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:09.546671 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:09.546684 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:09.548467 (+ 1783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:09.548942 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:09.548945 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:09.548946 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:09.549015 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:09.549354 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:09.549355 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:09.549355 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:09.549546 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:09.549736 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:09.549787 (+ 51us) server_negotiation.cc:299] Negotiation successful
0806 22:45:09.549891 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":164,"threads_started":1}
Aug 06 22:45:09 dist-test-slave-h3ft krb5kdc[27365](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056309, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.45.252@KRBTEST.COM for oryx/127.18.45.253@KRBTEST.COM
I20260806 22:45:09.557796 28342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:09.549650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55776 (local address 127.18.45.253:40509)
0806 22:45:09.549707 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:09.549710 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:09.550140 (+ 430us) server_negotiation.cc:407] Connection header received
0806 22:45:09.550316 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:09.550317 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:09.550339 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:09.550410 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:09.551288 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.551935 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.552666 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.552898 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.552956 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:09.555223 (+ 2267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:09.555224 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:09.555225 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:09.555233 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:09.556733 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:09.557089 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:09.557090 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:09.557090 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:09.557152 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:09.557323 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:09.557324 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:09.557324 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:09.557452 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:09.557507 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:09.557630 (+ 123us) server_negotiation.cc:299] Negotiation successful
0806 22:45:09.557712 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 22:45:09.557797 28453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:09.549565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.253:40509 (local address 127.0.0.1:55776)
0806 22:45:09.550046 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:09.550059 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:09.550229 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:09.550800 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:09.550801 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:09.551096 (+ 295us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:09.551186 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:09.551188 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.552029 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.552030 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:09.552588 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:09.552590 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.552982 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.552983 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:09.553061 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:09.553538 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:09.553543 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:09.555117 (+ 1574us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:45:09.556820 (+ 1703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:09.556822 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:09.556822 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:09.557020 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:09.557211 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:09.557211 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:09.557212 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:09.557272 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:09.557505 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:45:09.557506 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:45:09.557551 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 22:45:09.557605 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 22:45:09.557711 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460}
I20260806 22:45:09.558449 28451 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.45.252" port: 40869 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:45:09 dist-test-slave-h3ft krb5kdc[27365](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056309, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.45.252@KRBTEST.COM for oryx/127.18.45.252@KRBTEST.COM
I20260806 22:45:09.565815 28453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:09.558843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.252:40869 (local address 127.0.0.1:52736)
0806 22:45:09.558896 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:09.558908 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:09.558975 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:09.559235 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:09.559236 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:09.559388 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:09.559422 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:09.559423 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.560174 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.560176 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:09.560817 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:09.560819 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.561242 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.561243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:09.561296 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:09.561778 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:09.561784 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:09.563302 (+ 1518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:45:09.564919 (+ 1617us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:09.564920 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:09.564921 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:09.565100 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:09.565283 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:09.565283 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:09.565284 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:09.565341 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:09.565561 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:45:09.565562 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:45:09.565599 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 22:45:09.565651 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 22:45:09.565737 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 22:45:09.565881 28456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:09.558896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52736 (local address 127.18.45.252:40869)
0806 22:45:09.558956 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:09.558958 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:09.558968 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:45:09.559149 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:09.559150 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:09.559164 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:09.559232 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:09.559527 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.560088 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.560909 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.561155 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.561245 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:09.563403 (+ 2158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:09.563404 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:09.563405 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:09.563413 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:09.564836 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:09.565168 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:09.565169 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:09.565170 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:09.565224 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:09.565394 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:09.565394 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:09.565394 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:09.565508 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:09.565561 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:09.565682 (+ 121us) server_negotiation.cc:299] Negotiation successful
0806 22:45:09.565797 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":41}
I20260806 22:45:09.568029 28451 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d: Bootstrap starting.
I20260806 22:45:09.569145 28451 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d: Neither blocks nor log segments found. Creating new log.
I20260806 22:45:09.569533 28451 log.cc:824] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d: Log is configured to *not* fsync() on all Append() calls
I20260806 22:45:09.570526 28451 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d: No bootstrap required, opened a new log
I20260806 22:45:09.572773 28451 raft_consensus.cc:348] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39045 } } peers { permanent_uuid: "8f1bacf8b0b7450f8cff03a77e4c0890" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 40509 } } peers { permanent_uuid: "69b1737c4ee248689c79e5644bc1274d" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 40869 } }
I20260806 22:45:09.572964 28451 raft_consensus.cc:374] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:45:09.573031 28451 raft_consensus.cc:729] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69b1737c4ee248689c79e5644bc1274d, State: Initialized, Role: FOLLOWER
I20260806 22:45:09.573216 28451 consensus_queue.cc:260] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39045 } } peers { permanent_uuid: "8f1bacf8b0b7450f8cff03a77e4c0890" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 40509 } } peers { permanent_uuid: "69b1737c4ee248689c79e5644bc1274d" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 40869 } }
I20260806 22:45:09.573643 28451 sys_catalog.cc:564] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:45:09.573588 28458 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39045 } } peers { permanent_uuid: "8f1bacf8b0b7450f8cff03a77e4c0890" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 40509 } } peers { permanent_uuid: "69b1737c4ee248689c79e5644bc1274d" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 40869 } } }
I20260806 22:45:09.573767 28458 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d [sys.catalog]: This master's current role is: FOLLOWER
W20260806 22:45:09.576239 28474 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:45:09.576323 28474 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:45:09.579278 28349 master_runner.cc:428] Detected that this master 69b1737c4ee248689c79e5644bc1274d is joining an existing cluster
I20260806 22:45:09.579332 28349 master_runner.cc:432] Initiating AddMaster RPC to add 127.18.45.252:40869
I20260806 22:45:09.584583 27696 heartbeater.cc:503] Master 127.18.45.254:39045 was elected leader, sending a full tablet report...
I20260806 22:45:09.589192 28342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:09.580945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55792 (local address 127.18.45.253:40509)
0806 22:45:09.581032 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:09.581034 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:09.581045 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:45:09.581120 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:09.581121 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:09.581137 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:09.581198 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:09.581940 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.582858 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.584229 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.584524 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.584595 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:09.586173 (+ 1578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:09.586174 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:09.586174 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:09.586182 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:09.587727 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:09.588196 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:09.588198 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:09.588198 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:09.588286 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:09.588546 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:09.588547 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:09.588548 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:09.588696 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:09.588766 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:09.589015 (+ 249us) server_negotiation.cc:299] Negotiation successful
0806 22:45:09.589111 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 22:45:09.589217 28481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:09.580693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.253:40509 (local address 127.0.0.1:55792)
0806 22:45:09.580935 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:09.580953 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:09.581039 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:09.581276 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:09.581277 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:09.581694 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:09.581830 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:09.581834 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.582992 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.582994 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:09.584120 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:09.584123 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.584686 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.584687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:09.584755 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:09.585487 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:09.585493 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:09.586040 (+ 547us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:45:09.587818 (+ 1778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:09.587819 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:09.587820 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:09.588087 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:09.588376 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:09.588377 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:09.588377 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:09.588476 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:09.588769 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:45:09.588770 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:45:09.588857 (+ 87us) client_negotiation.cc:769] Sending connection context
0806 22:45:09.588942 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 22:45:09.589076 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"mutex_wait_us":57,"thread_start_us":115,"threads_started":1}
I20260806 22:45:09.602056 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 27519
I20260806 22:45:09.622372 28456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:09.581155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52738 (local address 127.18.45.252:40869)
0806 22:45:09.581198 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:09.581201 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:09.582049 (+ 848us) server_negotiation.cc:407] Connection header received
0806 22:45:09.582135 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:09.582137 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:09.582146 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:09.582174 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:09.582752 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.583567 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.584665 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.584973 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.585020 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:09.611918 (+ 26898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:09.611920 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:09.611921 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:09.611930 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:09.620260 (+ 8330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:09.620795 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:09.620797 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:09.620798 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:09.620855 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:09.621036 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:09.621036 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:09.621037 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:09.621523 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:09.621583 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:09.621799 (+ 216us) server_negotiation.cc:299] Negotiation successful
0806 22:45:09.622167 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 22:45:09.623348 28483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:09.581246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.252:40869 (local address 127.0.0.1:52738)
0806 22:45:09.581980 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:09.581992 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:09.582072 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:09.582230 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:09.582230 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:09.582581 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:09.582665 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:09.582668 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.583660 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.583660 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:09.584575 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:09.584578 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.585092 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.585093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:09.585174 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:09.611097 (+ 25923us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:09.611109 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:09.611781 (+ 672us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:45:09.620411 (+ 8630us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:09.620413 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:09.620414 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:09.620693 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:09.620922 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:09.620922 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:09.620923 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:09.620978 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:09.621632 (+ 654us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:45:09.621633 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:45:09.621689 (+ 56us) client_negotiation.cc:769] Sending connection context
0806 22:45:09.622872 (+ 1183us) client_negotiation.cc:240] Negotiation successful
0806 22:45:09.623173 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":714,"thread_start_us":93,"threads_started":1}
I20260806 22:45:09.624235 27504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:09.580772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56036 (local address 127.18.45.254:39045)
0806 22:45:09.580843 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:09.580847 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:09.581260 (+ 413us) server_negotiation.cc:407] Connection header received
0806 22:45:09.581374 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:09.581376 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:09.581392 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:09.581429 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:09.581971 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.582901 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.584277 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.584634 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.584678 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:09.620929 (+ 36251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:09.620932 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:09.620933 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:09.620944 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:09.622742 (+ 1798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:09.623318 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:09.623320 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:09.623320 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:09.623392 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:09.623621 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:09.623622 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:09.623623 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:09.623777 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:09.623832 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:09.624027 (+ 195us) server_negotiation.cc:299] Negotiation successful
0806 22:45:09.624123 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 22:45:09.624645 28482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:09.580727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39045 (local address 127.0.0.1:56036)
0806 22:45:09.581191 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:09.581202 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:09.581288 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:09.581510 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:09.581512 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:09.581771 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:09.581862 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:09.581864 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.583067 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.583068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:09.584172 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:09.584175 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.584762 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.584763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:09.584833 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:09.619361 (+ 34528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:09.619370 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:09.620771 (+ 1401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:45:09.622923 (+ 2152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:09.622924 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:09.622924 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:09.623217 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:09.623481 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:09.623482 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:09.623482 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:09.623548 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:09.623866 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:45:09.623867 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:45:09.623914 (+ 47us) client_negotiation.cc:769] Sending connection context
0806 22:45:09.624384 (+ 470us) client_negotiation.cc:240] Negotiation successful
0806 22:45:09.624528 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"mutex_wait_us":79,"thread_start_us":119,"threads_started":1}
I20260806 22:45:09.627569 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.193:37419
--local_ip_for_outbound_sockets=127.18.45.193
--tserver_master_addrs=127.18.45.254:39045
--webserver_port=36159
--webserver_interface=127.18.45.193
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/krb5kdc/oryx_127.18.45.193.keytab
--principal=oryx/127.18.45.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:33219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.18.45.254:39045,127.18.45.253:40509
--tserver_master_addrs=127.18.45.254:39045,127.18.45.253:40509,127.18.45.252:40869 with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:45:09.631721 28214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:09.627033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.253:40509 (local address 127.0.0.1:55802)
0806 22:45:09.627076 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:09.627090 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:09.627143 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:09.627450 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:09.627451 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:09.627454 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:45:09.627526 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:09.627528 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.628872 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.628873 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:09.630280 (+ 1407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:09.630284 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.631369 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.631370 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:09.631460 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:09.631489 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 22:45:09.631542 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 22:45:09.631605 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260806 22:45:09.632316 28342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:09.627197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55802 (local address 127.18.45.253:40509)
0806 22:45:09.627273 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:09.627276 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:09.627285 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:45:09.627321 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:09.627322 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:09.627357 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:09.627431 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:45:09.627629 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.628411 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.630405 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.631145 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.631215 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:09.631330 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:09.632142 (+ 812us) server_negotiation.cc:299] Negotiation successful
0806 22:45:09.632209 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 22:45:09.632786 28289 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
Aug 06 22:45:09 dist-test-slave-h3ft krb5kdc[27365](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056308, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.45.254@KRBTEST.COM for oryx/127.18.45.252@KRBTEST.COM
I20260806 22:45:09.633018 28327 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc26ea2c399a41b3857e7d7f2a8baef6. Latest consensus state: current_term: 1 leader_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39045 } } peers { permanent_uuid: "8f1bacf8b0b7450f8cff03a77e4c0890" member_type: NON_VOTER last_known_addr { host: "127.18.45.253" port: 40509 } attrs { promote: true } } }
I20260806 22:45:09.633162 28327 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890 [sys.catalog]: This master's current role is: LEARNER
I20260806 22:45:09.636880 28484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:09.627742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.252:40869 (local address 127.0.0.1:52748)
0806 22:45:09.628104 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:09.628115 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:09.628193 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:09.628394 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:09.628394 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:09.628618 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:09.628671 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:09.628673 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.629599 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.629600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:09.630406 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:09.630409 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.630951 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.630952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:09.631038 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:09.631545 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:09.631550 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:09.633276 (+ 1726us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:45:09.635571 (+ 2295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:09.635573 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:09.635573 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:09.635823 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:09.636070 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:09.636071 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:09.636071 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:09.636143 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:09.636476 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:45:09.636477 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:45:09.636532 (+ 55us) client_negotiation.cc:769] Sending connection context
0806 22:45:09.636613 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 22:45:09.636739 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":140,"threads_started":1}
I20260806 22:45:09.636915 28456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:09.627999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52748 (local address 127.18.45.252:40869)
0806 22:45:09.628037 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:09.628039 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:09.628169 (+ 130us) server_negotiation.cc:407] Connection header received
0806 22:45:09.628272 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:09.628273 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:09.628283 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:09.628314 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:09.628759 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.629492 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.630520 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.630821 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.630864 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:09.633410 (+ 2546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:09.633412 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:09.633413 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:09.633423 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:09.635479 (+ 2056us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:09.635912 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:09.635914 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:09.635915 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:09.635990 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:09.636218 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:09.636220 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:09.636220 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:09.636397 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:09.636484 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:09.636665 (+ 181us) server_negotiation.cc:299] Negotiation successful
0806 22:45:09.636769 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:45:09.637427 27431 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.18.45.252:40869: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "69b1737c4ee248689c79e5644bc1274d"
member_type: NON_VOTER
last_known_addr {
host: "127.18.45.252"
port: 40869
}
attrs {
promote: true
}
}
dest_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6"
cas_config_opid_index: 5
I20260806 22:45:09.637740 27431 consensus_queue.cc:237] T 00000000000000000000000000000000 P bc26ea2c399a41b3857e7d7f2a8baef6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39045 } } peers { permanent_uuid: "8f1bacf8b0b7450f8cff03a77e4c0890" member_type: NON_VOTER last_known_addr { host: "127.18.45.253" port: 40509 } attrs { promote: true } } peers { permanent_uuid: "69b1737c4ee248689c79e5644bc1274d" member_type: NON_VOTER last_known_addr { host: "127.18.45.252" port: 40869 } attrs { promote: true } }
I20260806 22:45:09.638193 28218 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bc26ea2c399a41b3857e7d7f2a8baef6 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 69b1737c4ee248689c79e5644bc1274d (127.18.45.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39045 } } peers { permanent_uuid: "8f1bacf8b0b7450f8cff03a77e4c0890" member_type: NON_VOTER last_known_addr { host: "127.18.45.253" port: 40509 } attrs { promote: true } } peers { permanent_uuid: "69b1737c4ee248689c79e5644bc1274d" member_type: NON_VOTER last_known_addr { host: "127.18.45.252" port: 40869 } attrs { promote: true } } }
I20260806 22:45:09.638509 28408 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:45:09.639575 28218 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.18.45.252:40869
I20260806 22:45:09.639669 28218 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc26ea2c399a41b3857e7d7f2a8baef6 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39045 } } peers { permanent_uuid: "8f1bacf8b0b7450f8cff03a77e4c0890" member_type: NON_VOTER last_known_addr { host: "127.18.45.253" port: 40509 } attrs { promote: true } } peers { permanent_uuid: "69b1737c4ee248689c79e5644bc1274d" member_type: NON_VOTER last_known_addr { host: "127.18.45.252" port: 40869 } attrs { promote: true } } }
I20260806 22:45:09.639755 28218 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc26ea2c399a41b3857e7d7f2a8baef6 [sys.catalog]: This master's current role is: LEADER
I20260806 22:45:09.639750 28408 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d [term 1 FOLLOWER]: Refusing update from remote peer bc26ea2c399a41b3857e7d7f2a8baef6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 22:45:09.640110 28349 master.cc:537] Master@127.18.45.252:40869 shutting down...
I20260806 22:45:09.640312 28289 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890 [term 1 LEARNER]: Refusing update from remote peer bc26ea2c399a41b3857e7d7f2a8baef6: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 22:45:09.640543 28216 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bc26ea2c399a41b3857e7d7f2a8baef6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "69b1737c4ee248689c79e5644bc1274d" member_type: NON_VOTER last_known_addr { host: "127.18.45.252" port: 40869 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:45:09.640815 28216 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bc26ea2c399a41b3857e7d7f2a8baef6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f1bacf8b0b7450f8cff03a77e4c0890" member_type: NON_VOTER last_known_addr { host: "127.18.45.253" port: 40509 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260806 22:45:09.641425 27413 consensus_peers.cc:487] T 00000000000000000000000000000000 P bc26ea2c399a41b3857e7d7f2a8baef6 -> Peer 69b1737c4ee248689c79e5644bc1274d (127.18.45.252:40869): Couldn't send request to peer 69b1737c4ee248689c79e5644bc1274d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 22:45:09.642021 28289 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 69b1737c4ee248689c79e5644bc1274d (127.18.45.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39045 } } peers { permanent_uuid: "8f1bacf8b0b7450f8cff03a77e4c0890" member_type: NON_VOTER last_known_addr { host: "127.18.45.253" port: 40509 } attrs { promote: true } } peers { permanent_uuid: "69b1737c4ee248689c79e5644bc1274d" member_type: NON_VOTER last_known_addr { host: "127.18.45.252" port: 40869 } attrs { promote: true } } }
I20260806 22:45:09.643626 28327 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39045 } } peers { permanent_uuid: "8f1bacf8b0b7450f8cff03a77e4c0890" member_type: NON_VOTER last_known_addr { host: "127.18.45.253" port: 40509 } attrs { promote: true } } peers { permanent_uuid: "69b1737c4ee248689c79e5644bc1274d" member_type: NON_VOTER last_known_addr { host: "127.18.45.252" port: 40869 } attrs { promote: true } } }
I20260806 22:45:09.643754 28327 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890 [sys.catalog]: This master's current role is: LEARNER
I20260806 22:45:09.644338 28218 raft_consensus.cc:1050] T 00000000000000000000000000000000 P bc26ea2c399a41b3857e7d7f2a8baef6: attempting to promote NON_VOTER 8f1bacf8b0b7450f8cff03a77e4c0890 to VOTER
I20260806 22:45:09.644605 28218 consensus_queue.cc:237] T 00000000000000000000000000000000 P bc26ea2c399a41b3857e7d7f2a8baef6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39045 } } peers { permanent_uuid: "8f1bacf8b0b7450f8cff03a77e4c0890" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 40509 } attrs { promote: false } } peers { permanent_uuid: "69b1737c4ee248689c79e5644bc1274d" member_type: NON_VOTER last_known_addr { host: "127.18.45.252" port: 40869 } attrs { promote: true } }
I20260806 22:45:09.644749 28349 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:45:09.644852 28349 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:45:09.644904 28349 tablet_replica.cc:330] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d: stopping tablet replica
I20260806 22:45:09.645324 28289 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890 [term 1 LEARNER]: Refusing update from remote peer bc26ea2c399a41b3857e7d7f2a8baef6: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260806 22:45:09.645558 28489 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bc26ea2c399a41b3857e7d7f2a8baef6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f1bacf8b0b7450f8cff03a77e4c0890" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 40509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260806 22:45:09.645668 27413 consensus_peers.cc:487] T 00000000000000000000000000000000 P bc26ea2c399a41b3857e7d7f2a8baef6 -> Peer 69b1737c4ee248689c79e5644bc1274d (127.18.45.252:40869): Couldn't send request to peer 69b1737c4ee248689c79e5644bc1274d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 22:45:09.646225 28216 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bc26ea2c399a41b3857e7d7f2a8baef6 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 8f1bacf8b0b7450f8cff03a77e4c0890 (127.18.45.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39045 } } peers { permanent_uuid: "8f1bacf8b0b7450f8cff03a77e4c0890" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 40509 } attrs { promote: false } } peers { permanent_uuid: "69b1737c4ee248689c79e5644bc1274d" member_type: NON_VOTER last_known_addr { host: "127.18.45.252" port: 40869 } attrs { promote: true } } }
I20260806 22:45:09.647353 28216 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc26ea2c399a41b3857e7d7f2a8baef6 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39045 } } peers { permanent_uuid: "8f1bacf8b0b7450f8cff03a77e4c0890" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 40509 } attrs { promote: false } } peers { permanent_uuid: "69b1737c4ee248689c79e5644bc1274d" member_type: NON_VOTER last_known_addr { host: "127.18.45.252" port: 40869 } attrs { promote: true } } }
I20260806 22:45:09.647437 28216 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc26ea2c399a41b3857e7d7f2a8baef6 [sys.catalog]: This master's current role is: LEADER
I20260806 22:45:09.647492 28216 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc26ea2c399a41b3857e7d7f2a8baef6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39045 } } peers { permanent_uuid: "8f1bacf8b0b7450f8cff03a77e4c0890" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 40509 } attrs { promote: false } } peers { permanent_uuid: "69b1737c4ee248689c79e5644bc1274d" member_type: NON_VOTER last_known_addr { host: "127.18.45.252" port: 40869 } attrs { promote: true } } }
I20260806 22:45:09.647531 28216 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc26ea2c399a41b3857e7d7f2a8baef6 [sys.catalog]: This master's current role is: LEADER
I20260806 22:45:09.648058 28289 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 8f1bacf8b0b7450f8cff03a77e4c0890 (127.18.45.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39045 } } peers { permanent_uuid: "8f1bacf8b0b7450f8cff03a77e4c0890" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 40509 } attrs { promote: false } } peers { permanent_uuid: "69b1737c4ee248689c79e5644bc1274d" member_type: NON_VOTER last_known_addr { host: "127.18.45.252" port: 40869 } attrs { promote: true } } }
I20260806 22:45:09.649159 28327 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39045 } } peers { permanent_uuid: "8f1bacf8b0b7450f8cff03a77e4c0890" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 40509 } attrs { promote: false } } peers { permanent_uuid: "69b1737c4ee248689c79e5644bc1274d" member_type: NON_VOTER last_known_addr { host: "127.18.45.252" port: 40869 } attrs { promote: true } } }
I20260806 22:45:09.649255 28327 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:45:09.659559 28349 master.cc:552] Master@127.18.45.252:40869 shutdown complete.
I20260806 22:45:09.659660 28349 master_runner.cc:305] Clearing existing system tablet
I20260806 22:45:09.659894 28349 fs_manager.cc:359] Metadata directory not provided
I20260806 22:45:09.659969 28349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:45:09.661051 28349 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:45:09.661741 28491 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:09.661980 28349 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:45:09.662137 28349 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:45:09.662185 28349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/master-2/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/master-2/wal
uuid: "69b1737c4ee248689c79e5644bc1274d"
format_stamp: "Formatted at 2026-08-06 22:45:09 on dist-test-slave-h3ft"
server_key: "b1621c0123e4751e846ec93a491e176d"
server_key_iv: "2b96ab35c6718477644b1a6c67a313b6"
server_key_version: "encryptionkey@0"
I20260806 22:45:09.663494 28349 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:45:09.665324 28349 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 22:45:09.665434 28349 log.cc:1196] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d: Deleting WAL directory at /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260806 22:45:09.665695 28349 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d: Deleting consensus metadata
I20260806 22:45:09.665835 28349 master_runner.cc:315] Copying system tablet from 127.18.45.254:39045
I20260806 22:45:09.666680 28349 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d: tablet copy: Beginning tablet copy session from remote peer at address 127.18.45.254:39045
I20260806 22:45:09.679133 28498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:09.667214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39045 (local address 127.0.0.1:56048)
0806 22:45:09.671142 (+ 3928us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:09.671162 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:09.671293 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:09.671491 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:09.671492 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:09.671765 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:09.671840 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:09.671843 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.672893 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.672894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:09.673799 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:09.673803 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.674310 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.674311 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:09.674390 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:09.675169 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:09.675175 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:09.675750 (+ 575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:45:09.677816 (+ 2066us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:09.677818 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:09.677818 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:09.678078 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:09.678314 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:09.678315 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:09.678315 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:09.678401 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:09.678697 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:45:09.678698 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:45:09.678758 (+ 60us) client_negotiation.cc:769] Sending connection context
0806 22:45:09.678835 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 22:45:09.678967 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3891,"thread_start_us":93,"threads_started":1}
I20260806 22:45:09.679116 27504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:09.667331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56048 (local address 127.18.45.254:39045)
0806 22:45:09.667411 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:09.667414 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:09.671285 (+ 3871us) server_negotiation.cc:407] Connection header received
0806 22:45:09.671370 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:09.671372 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:09.671392 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:09.671473 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:09.671935 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.672782 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.673889 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.674209 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.674280 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:09.675851 (+ 1571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:09.675853 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:09.675854 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:09.675863 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:09.677726 (+ 1863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:09.678163 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:09.678164 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:09.678165 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:09.678241 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:09.678470 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:09.678470 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:09.678471 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:09.678625 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:09.678697 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:09.678876 (+ 179us) server_negotiation.cc:299] Negotiation successful
0806 22:45:09.678970 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 22:45:09.679975 27461 tablet_copy_service.cc:140] P bc26ea2c399a41b3857e7d7f2a8baef6: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 69b1737c4ee248689c79e5644bc1274d ({username='oryx', principal='oryx/127.18.45.252@KRBTEST.COM'} at 127.0.0.1:56048)
I20260806 22:45:09.680063 27461 tablet_copy_service.cc:161] P bc26ea2c399a41b3857e7d7f2a8baef6: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 69b1737c4ee248689c79e5644bc1274d at {username='oryx', principal='oryx/127.18.45.252@KRBTEST.COM'} at 127.0.0.1:56048: session id = 69b1737c4ee248689c79e5644bc1274d-00000000000000000000000000000000
I20260806 22:45:09.680886 27461 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P bc26ea2c399a41b3857e7d7f2a8baef6: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 22:45:09.681589 28349 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 22:45:09.683833 28349 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d: tablet copy: Starting download of 0 data blocks...
I20260806 22:45:09.684116 28349 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d: tablet copy: Starting download of 1 WAL segments...
I20260806 22:45:09.685067 28349 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 22:45:09.686328 27461 tablet_copy_service.cc:342] P bc26ea2c399a41b3857e7d7f2a8baef6: Request end of tablet copy session 69b1737c4ee248689c79e5644bc1274d-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.18.45.252@KRBTEST.COM'} at 127.0.0.1:56048
I20260806 22:45:09.686398 27461 tablet_copy_service.cc:433] P bc26ea2c399a41b3857e7d7f2a8baef6: ending tablet copy session 69b1737c4ee248689c79e5644bc1274d-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 69b1737c4ee248689c79e5644bc1274d
I20260806 22:45:09.689905 28349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:45:09.691411 28499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:09.692109 28500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:09.692297 28503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:09.699991 28349 server_base.cc:1034] running on GCE node
I20260806 22:45:09.700182 28349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:45:09.700388 28349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:45:09.701581 28349 hybrid_clock.cc:648] HybridClock initialized: now 1786056309701552 us; error 45 us; skew 500 ppm
Aug 06 22:45:09 dist-test-slave-h3ft krb5kdc[27365](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056309, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.45.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:45:09.703593 28349 init.cc:377] Logged in from keytab as oryx/127.18.45.252@KRBTEST.COM (short username oryx)
I20260806 22:45:09.703954 28349 webserver.cc:460] Webserver started at http://127.18.45.252:38821/ using document root <none> and password file <none>
I20260806 22:45:09.704102 28349 fs_manager.cc:359] Metadata directory not provided
I20260806 22:45:09.704140 28349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:45:09.705135 28349 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:45:09.705787 28518 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:09.705907 28349 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:09.705943 28349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/master-2/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/master-2/wal
uuid: "69b1737c4ee248689c79e5644bc1274d"
format_stamp: "Formatted at 2026-08-06 22:45:09 on dist-test-slave-h3ft"
server_key: "b1621c0123e4751e846ec93a491e176d"
server_key_iv: "2b96ab35c6718477644b1a6c67a313b6"
server_key_version: "encryptionkey@0"
I20260806 22:45:09.706037 28349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:45:09.713241 28349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:45:09.715433 28349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:45:09.719588 28349 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.252:40869
I20260806 22:45:09.720032 28349 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/master-2/data/info.pb
I20260806 22:45:09.721227 28593 sys_catalog.cc:263] Verifying existing consensus state
I20260806 22:45:09.721839 28593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d: Bootstrap starting.
I20260806 22:45:09.723201 28592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.252:40869 every 8 connection(s)
I20260806 22:45:09.725672 28593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:45:09.725914 28593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d: Bootstrap complete.
I20260806 22:45:09.726186 28593 raft_consensus.cc:348] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39045 } } peers { permanent_uuid: "8f1bacf8b0b7450f8cff03a77e4c0890" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 40509 } attrs { promote: false } } peers { permanent_uuid: "69b1737c4ee248689c79e5644bc1274d" member_type: NON_VOTER last_known_addr { host: "127.18.45.252" port: 40869 } attrs { promote: true } }
I20260806 22:45:09.726296 28593 raft_consensus.cc:729] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 69b1737c4ee248689c79e5644bc1274d, State: Initialized, Role: LEARNER
I20260806 22:45:09.726348 28593 consensus_queue.cc:260] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39045 } } peers { permanent_uuid: "8f1bacf8b0b7450f8cff03a77e4c0890" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 40509 } attrs { promote: false } } peers { permanent_uuid: "69b1737c4ee248689c79e5644bc1274d" member_type: NON_VOTER last_known_addr { host: "127.18.45.252" port: 40869 } attrs { promote: true } }
I20260806 22:45:09.726634 28593 sys_catalog.cc:564] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:45:09.726606 28597 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39045 } } peers { permanent_uuid: "8f1bacf8b0b7450f8cff03a77e4c0890" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 40509 } attrs { promote: false } } peers { permanent_uuid: "69b1737c4ee248689c79e5644bc1274d" member_type: NON_VOTER last_known_addr { host: "127.18.45.252" port: 40869 } attrs { promote: true } } }
I20260806 22:45:09.726923 28597 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d [sys.catalog]: This master's current role is: LEARNER
I20260806 22:45:09.729085 28615 catalog_manager.cc:1259] Loaded cluster ID: f71c30cbb9674010be9e9985b316b369
I20260806 22:45:09.729156 28615 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d: loading cluster ID for follower catalog manager: success
I20260806 22:45:09.730365 28615 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d: acquiring CA information for follower catalog manager: success
I20260806 22:45:09.730747 28615 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 22:45:09.751055 28486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:45:09.751394 28486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:45:09.751444 28486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:45:09.752987 28486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:45:09.753067 28486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:45:09.753118 28486 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.193
I20260806 22:45:09.755352 28486 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:33219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/krb5kdc/oryx_127.18.45.193.keytab
--principal=oryx/127.18.45.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.193:37419
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.45.193
--webserver_port=36159
--tserver_master_addrs=127.18.45.254:39045,127.18.45.253:40509,127.18.45.252:40869
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28486
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.193
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:45:09.755676 28486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:45:09.755990 28486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:45:09.763898 28627 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:09.763921 28624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:09.763945 28622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:09.764330 28486 server_base.cc:1034] running on GCE node
I20260806 22:45:09.764564 28486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:45:09.764848 28486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:45:09.766000 28486 hybrid_clock.cc:648] HybridClock initialized: now 1786056309765979 us; error 38 us; skew 500 ppm
Aug 06 22:45:09 dist-test-slave-h3ft krb5kdc[27365](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056309, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.45.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:45:09.769111 28486 init.cc:377] Logged in from keytab as oryx/127.18.45.193@KRBTEST.COM (short username oryx)
I20260806 22:45:09.769397 27889 heartbeater.cc:503] Master 127.18.45.254:39045 was elected leader, sending a full tablet report...
I20260806 22:45:09.769781 28486 webserver.cc:460] Webserver started at http://127.18.45.193:36159/ using document root <none> and password file <none>
I20260806 22:45:09.770126 28486 fs_manager.cc:359] Metadata directory not provided
I20260806 22:45:09.770227 28486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:45:09.771948 28486 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:45:09.772724 28636 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:09.772986 28486 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:45:09.773109 28486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-0/wal
uuid: "0bc1abeccf9a48f7b80fcde36b6c6a1e"
format_stamp: "Formatted at 2026-08-06 22:45:08 on dist-test-slave-h3ft"
server_key: "1e79d8b1103916a4e1a4bd2d229d296b"
server_key_iv: "0fd99256739aeede5ac3ee8a3898742e"
server_key_version: "encryptionkey@0"
I20260806 22:45:09.773537 28486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:45:09.791770 28486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:45:09.795073 28486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:45:09.795226 28486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:45:09.795544 28486 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:45:09.795972 28486 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:45:09.796015 28486 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:09.796066 28486 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:45:09.796093 28486 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:09.805233 28486 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.193:37419
I20260806 22:45:09.805305 28794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.193:37419 every 8 connection(s)
I20260806 22:45:09.805812 28486 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
I20260806 22:45:09.806792 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 28486
I20260806 22:45:09.807049 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 27704
Aug 06 22:45:09 dist-test-slave-h3ft krb5kdc[27365](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056309, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.45.193@KRBTEST.COM for oryx/127.18.45.253@KRBTEST.COM
I20260806 22:45:09.832656 28342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:09.807459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:35821 (local address 127.18.45.253:40509)
0806 22:45:09.813513 (+ 6054us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:09.813516 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:09.816243 (+ 2727us) server_negotiation.cc:407] Connection header received
0806 22:45:09.817990 (+ 1747us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:09.817992 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:09.818014 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:09.818194 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:09.820014 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.821285 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.823850 (+ 2565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.824331 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.824397 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:09.828813 (+ 4416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:09.828815 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:09.828816 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:09.828827 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:09.830790 (+ 1963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:09.831370 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:09.831372 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:09.831372 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:09.831444 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:09.831772 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:09.831773 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:09.831773 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:09.831955 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:09.832016 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:09.832424 (+ 408us) server_negotiation.cc:299] Negotiation successful
0806 22:45:09.832529 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5999}
I20260806 22:45:09.833074 28803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:09.806734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.253:40509 (local address 127.18.45.193:35821)
0806 22:45:09.815997 (+ 9263us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:09.816080 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:09.817835 (+ 1755us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:09.818435 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:09.818444 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:09.819269 (+ 825us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:09.819818 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:09.819829 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.821458 (+ 1629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.821459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:09.823699 (+ 2240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:09.823705 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.824543 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.824545 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:09.824795 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:09.825908 (+ 1113us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:09.825971 (+ 63us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:09.828690 (+ 2719us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:45:09.830891 (+ 2201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:09.830900 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:09.830904 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:09.831281 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:09.831509 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:09.831510 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:09.831510 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:09.831682 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:09.832054 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:45:09.832061 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:45:09.832259 (+ 198us) client_negotiation.cc:769] Sending connection context
0806 22:45:09.832388 (+ 129us) client_negotiation.cc:240] Negotiation successful
0806 22:45:09.832605 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9199,"thread_start_us":116,"threads_started":1}
Aug 06 22:45:09 dist-test-slave-h3ft krb5kdc[27365](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056309, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.45.193@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM
I20260806 22:45:09.833982 28798 heartbeater.cc:347] Connected to a master server at 127.18.45.253:40509
I20260806 22:45:09.834125 28798 heartbeater.cc:464] Registering TS with master...
Aug 06 22:45:09 dist-test-slave-h3ft krb5kdc[27365](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056309, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.45.193@KRBTEST.COM for oryx/127.18.45.252@KRBTEST.COM
I20260806 22:45:09.834401 28798 heartbeater.cc:511] Master 127.18.45.253:40509 requested a full tablet report, sending...
I20260806 22:45:09.835263 28274 ts_manager.cc:194] Registered new tserver with Master: 0bc1abeccf9a48f7b80fcde36b6c6a1e (127.18.45.193:37419)
I20260806 22:45:09.836736 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.194:43321
--local_ip_for_outbound_sockets=127.18.45.194
--tserver_master_addrs=127.18.45.254:39045
--webserver_port=40585
--webserver_interface=127.18.45.194
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/krb5kdc/oryx_127.18.45.194.keytab
--principal=oryx/127.18.45.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:33219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.18.45.254:39045,127.18.45.253:40509
--tserver_master_addrs=127.18.45.254:39045,127.18.45.253:40509,127.18.45.252:40869 with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:45:09.837250 27504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:09.814867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:38531 (local address 127.18.45.254:39045)
0806 22:45:09.814959 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:09.814962 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:09.825781 (+ 10819us) server_negotiation.cc:407] Connection header received
0806 22:45:09.827295 (+ 1514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:09.827297 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:09.827315 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:09.827355 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:09.828602 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.829422 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.830713 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.831110 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.831160 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:09.833965 (+ 2805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:09.833967 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:09.833968 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:09.833980 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:09.835826 (+ 1846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:09.836258 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:09.836259 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:09.836260 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:09.836325 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:09.836541 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:09.836542 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:09.836542 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:09.836707 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:09.836790 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:09.837008 (+ 218us) server_negotiation.cc:299] Negotiation successful
0806 22:45:09.837107 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 22:45:09.837759 28800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:09.806735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39045 (local address 127.18.45.193:38531)
0806 22:45:09.825660 (+ 18925us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:09.825675 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:09.827174 (+ 1499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:09.827436 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:09.827436 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:09.827832 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:09.827939 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:09.827942 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.829622 (+ 1680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.829623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:09.830602 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:09.830607 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.831258 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.831259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:09.831338 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:09.832040 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:09.832046 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:09.833852 (+ 1806us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:45:09.835911 (+ 2059us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:09.835912 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:09.835913 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:09.836165 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:09.836400 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:09.836401 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:09.836401 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:09.836458 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:09.836796 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:45:09.836797 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:45:09.836852 (+ 55us) client_negotiation.cc:769] Sending connection context
0806 22:45:09.837396 (+ 544us) client_negotiation.cc:240] Negotiation successful
0806 22:45:09.837579 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18881,"mutex_wait_us":3,"thread_start_us":129,"threads_started":1}
I20260806 22:45:09.838156 28797 heartbeater.cc:347] Connected to a master server at 127.18.45.254:39045
I20260806 22:45:09.838215 28797 heartbeater.cc:464] Registering TS with master...
I20260806 22:45:09.838303 28797 heartbeater.cc:511] Master 127.18.45.254:39045 requested a full tablet report, sending...
I20260806 22:45:09.838644 27431 ts_manager.cc:194] Re-registered known tserver with Master: 0bc1abeccf9a48f7b80fcde36b6c6a1e (127.18.45.193:37419)
I20260806 22:45:09.838822 28802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:09.806867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:34891 (local address 127.18.45.252:40869)
0806 22:45:09.827136 (+ 20269us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:09.827139 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:09.827150 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:45:09.827189 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:09.827190 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:09.827209 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:09.827294 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:09.829017 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.830179 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.831287 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.831612 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.831688 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:09.834804 (+ 3116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:09.834806 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:09.834807 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:09.834817 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:09.837270 (+ 2453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:09.837736 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:09.837738 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:09.837738 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:09.837831 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:09.838053 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:09.838054 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:09.838055 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:09.838206 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:09.838274 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:09.838550 (+ 276us) server_negotiation.cc:299] Negotiation successful
0806 22:45:09.838655 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20233,"thread_start_us":81,"threads_started":1}
I20260806 22:45:09.838917 28801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:09.806734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.252:40869 (local address 127.18.45.193:34891)
0806 22:45:09.826854 (+ 20120us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:09.826869 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:09.826950 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:09.827364 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:09.827364 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:09.828092 (+ 728us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:09.828892 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:09.828894 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.830291 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.830292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:09.831187 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:09.831190 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.831702 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.831703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:09.831784 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:09.833129 (+ 1345us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:09.833134 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:09.834697 (+ 1563us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:45:09.837405 (+ 2708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:09.837407 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:09.837407 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:09.837657 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:09.837922 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:09.837924 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:09.837924 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:09.837986 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:09.838300 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:45:09.838301 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:45:09.838363 (+ 62us) client_negotiation.cc:769] Sending connection context
0806 22:45:09.838433 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 22:45:09.838560 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20080,"mutex_wait_us":318,"thread_start_us":86,"threads_started":1}
I20260806 22:45:09.839257 27431 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.45.193@KRBTEST.COM'} at 127.18.45.193:38531
I20260806 22:45:09.839684 28795 heartbeater.cc:347] Connected to a master server at 127.18.45.252:40869
I20260806 22:45:09.839758 28795 heartbeater.cc:464] Registering TS with master...
I20260806 22:45:09.839840 28795 heartbeater.cc:511] Master 127.18.45.252:40869 requested a full tablet report, sending...
I20260806 22:45:09.840301 28539 ts_manager.cc:194] Registered new tserver with Master: 0bc1abeccf9a48f7b80fcde36b6c6a1e (127.18.45.193:37419)
W20260806 22:45:09.936348 28806 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:45:09.936630 28806 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:45:09.936661 28806 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:45:09.937909 28806 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:45:09.937951 28806 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:45:09.937980 28806 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.194
I20260806 22:45:09.940140 28806 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:33219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/krb5kdc/oryx_127.18.45.194.keytab
--principal=oryx/127.18.45.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.194:43321
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.45.194
--webserver_port=40585
--tserver_master_addrs=127.18.45.254:39045,127.18.45.253:40509,127.18.45.252:40869
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28806
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.194
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:45:09.940418 28806 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:45:09.940687 28806 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:45:09.947044 28815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:09.947144 28819 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:09.947034 28816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:09.947495 28806 server_base.cc:1034] running on GCE node
I20260806 22:45:09.947772 28806 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:45:09.948078 28806 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:45:09.949254 28806 hybrid_clock.cc:648] HybridClock initialized: now 1786056309949231 us; error 41 us; skew 500 ppm
Aug 06 22:45:09 dist-test-slave-h3ft krb5kdc[27365](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056309, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.45.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:45:09.952557 28806 init.cc:377] Logged in from keytab as oryx/127.18.45.194@KRBTEST.COM (short username oryx)
I20260806 22:45:09.953152 28806 webserver.cc:460] Webserver started at http://127.18.45.194:40585/ using document root <none> and password file <none>
I20260806 22:45:09.953424 28806 fs_manager.cc:359] Metadata directory not provided
I20260806 22:45:09.953523 28806 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:45:09.955268 28806 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:45:09.956097 28828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:09.956333 28806 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:45:09.956447 28806 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-1/wal
uuid: "845d3ece65974e25ad6ce260ba27b693"
format_stamp: "Formatted at 2026-08-06 22:45:08 on dist-test-slave-h3ft"
server_key: "743eaa686c6789ac2bd6efc8fba723b9"
server_key_iv: "ec990f89046de94fdccf720ee6b38108"
server_key_version: "encryptionkey@0"
I20260806 22:45:09.956876 28806 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:45:09.971246 28806 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:45:09.971328 28074 heartbeater.cc:503] Master 127.18.45.254:39045 was elected leader, sending a full tablet report...
I20260806 22:45:09.974129 28806 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:45:09.974331 28806 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:45:09.974644 28806 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:45:09.975068 28806 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:45:09.975109 28806 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:09.975135 28806 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:45:09.975174 28806 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:09.982626 28806 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.194:43321
I20260806 22:45:09.982734 28996 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.194:43321 every 8 connection(s)
I20260806 22:45:09.983250 28806 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
I20260806 22:45:09.984632 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 28806
I20260806 22:45:09.984838 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 27897
Aug 06 22:45:09 dist-test-slave-h3ft krb5kdc[27365](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056309, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.45.194@KRBTEST.COM for oryx/127.18.45.252@KRBTEST.COM
I20260806 22:45:09.997632 28802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:09.984665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:40419 (local address 127.18.45.252:40869)
0806 22:45:09.984746 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:09.984749 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:09.984926 (+ 177us) server_negotiation.cc:407] Connection header received
0806 22:45:09.986493 (+ 1567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:09.986496 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:09.986517 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:09.986551 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:09.987812 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.988559 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.990043 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.990346 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.990389 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:09.993649 (+ 3260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:09.993652 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:09.993652 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:09.993663 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:09.995495 (+ 1832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:09.996039 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:09.996041 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:09.996041 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:09.996105 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:09.996366 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:09.996367 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:09.996368 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:09.996617 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:09.996685 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:09.997267 (+ 582us) server_negotiation.cc:299] Negotiation successful
0806 22:45:09.997430 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 22:45:09.998562 29006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:09.984585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.252:40869 (local address 127.18.45.194:40419)
0806 22:45:09.984843 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:09.984852 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:09.986350 (+ 1498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:09.986928 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:09.986933 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:09.987390 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:09.987664 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:09.987670 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.988689 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.988690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:09.989941 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:09.989945 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.990501 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.990502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:09.990665 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:09.991407 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:09.991420 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:09.993521 (+ 2101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:45:09.995604 (+ 2083us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:09.995614 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:09.995618 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:09.995959 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:09.996169 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:09.996170 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:09.996170 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:09.996304 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:09.996753 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:45:09.996763 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:45:09.997034 (+ 271us) client_negotiation.cc:769] Sending connection context
0806 22:45:09.997801 (+ 767us) client_negotiation.cc:240] Negotiation successful
0806 22:45:09.998113 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"mutex_wait_us":1,"thread_start_us":73,"threads_started":1}
I20260806 22:45:09.999366 28998 heartbeater.cc:347] Connected to a master server at 127.18.45.252:40869
I20260806 22:45:09.999491 28998 heartbeater.cc:464] Registering TS with master...
I20260806 22:45:09.999744 28998 heartbeater.cc:511] Master 127.18.45.252:40869 requested a full tablet report, sending...
I20260806 22:45:10.000366 28539 ts_manager.cc:194] Registered new tserver with Master: 845d3ece65974e25ad6ce260ba27b693 (127.18.45.194:43321)
Aug 06 22:45:09 dist-test-slave-h3ft krb5kdc[27365](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056309, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.45.194@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM
Aug 06 22:45:10 dist-test-slave-h3ft krb5kdc[27365](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056309, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.45.194@KRBTEST.COM for oryx/127.18.45.253@KRBTEST.COM
I20260806 22:45:10.008086 27504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:09.984226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:50555 (local address 127.18.45.254:39045)
0806 22:45:09.984281 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:09.984283 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:09.997258 (+ 12975us) server_negotiation.cc:407] Connection header received
0806 22:45:09.997383 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:09.997385 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:09.997402 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:09.997433 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:09.998091 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.998798 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:10.000989 (+ 2191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:10.001288 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:10.001356 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:10.004401 (+ 3045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:10.004404 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:10.004404 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:10.004416 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:10.006644 (+ 2228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:10.007190 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:10.007192 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:10.007193 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:10.007260 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:10.007436 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:10.007436 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:10.007437 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:10.007590 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:10.007652 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:10.007830 (+ 178us) server_negotiation.cc:299] Negotiation successful
0806 22:45:10.007924 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 22:45:10.008836 29005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:09.984173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39045 (local address 127.18.45.194:50555)
0806 22:45:09.984521 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:09.984576 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:09.997308 (+ 12732us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:09.997584 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:09.997585 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:09.997898 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:09.997994 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:09.997996 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:09.999027 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:09.999028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:10.000859 (+ 1831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:10.000863 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:10.001424 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:10.001425 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:10.001512 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:10.002087 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:10.002093 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:10.004275 (+ 2182us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:45:10.006782 (+ 2507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:10.006784 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:10.006784 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:10.007102 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:10.007323 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:10.007323 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:10.007324 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:10.007379 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:10.007681 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:45:10.007682 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:45:10.007732 (+ 50us) client_negotiation.cc:769] Sending connection context
0806 22:45:10.008227 (+ 495us) client_negotiation.cc:240] Negotiation successful
0806 22:45:10.008655 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"mutex_wait_us":5,"thread_start_us":119,"threads_started":1}
I20260806 22:45:10.009467 28999 heartbeater.cc:347] Connected to a master server at 127.18.45.254:39045
I20260806 22:45:10.009531 28999 heartbeater.cc:464] Registering TS with master...
I20260806 22:45:10.009636 28999 heartbeater.cc:511] Master 127.18.45.254:39045 requested a full tablet report, sending...
I20260806 22:45:10.009907 27431 ts_manager.cc:194] Re-registered known tserver with Master: 845d3ece65974e25ad6ce260ba27b693 (127.18.45.194:43321)
I20260806 22:45:10.010560 27431 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.45.194@KRBTEST.COM'} at 127.18.45.194:50555
I20260806 22:45:10.012004 28342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:09.984172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:33123 (local address 127.18.45.253:40509)
0806 22:45:09.984243 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:09.984246 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:09.984941 (+ 695us) server_negotiation.cc:407] Connection header received
0806 22:45:09.999373 (+ 14432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:09.999375 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:09.999394 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:09.999462 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:10.001188 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:10.002228 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:10.003624 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:10.004033 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:10.004095 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:10.007870 (+ 3775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:10.007873 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:10.007873 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:10.007886 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:10.010129 (+ 2243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:10.010729 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:10.010732 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:10.010732 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:10.010822 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:10.011178 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:10.011179 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:10.011179 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:10.011392 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:10.011468 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:10.011738 (+ 270us) server_negotiation.cc:299] Negotiation successful
0806 22:45:10.011858 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 22:45:10.012137 29003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:09.984173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.253:40509 (local address 127.18.45.194:33123)
0806 22:45:09.984429 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:09.984824 (+ 395us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:09.997491 (+ 12667us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:09.999595 (+ 2104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:09.999597 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:09.999883 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:10.001047 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:10.001049 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:10.002366 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:10.002367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:10.003464 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:10.003469 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:10.004213 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:10.004215 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:10.004315 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:10.005108 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:10.005115 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:10.007700 (+ 2585us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:45:10.010267 (+ 2567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:10.010269 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:10.010270 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:10.010605 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:10.010919 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:10.010920 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:10.010921 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:10.011061 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:10.011512 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:45:10.011513 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:45:10.011591 (+ 78us) client_negotiation.cc:769] Sending connection context
0806 22:45:10.011677 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 22:45:10.011868 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"mutex_wait_us":1072,"thread_start_us":121,"threads_started":1}
I20260806 22:45:10.012544 29001 heartbeater.cc:347] Connected to a master server at 127.18.45.253:40509
I20260806 22:45:10.012612 29001 heartbeater.cc:464] Registering TS with master...
I20260806 22:45:10.012748 29001 heartbeater.cc:511] Master 127.18.45.253:40509 requested a full tablet report, sending...
I20260806 22:45:10.013149 28274 ts_manager.cc:194] Registered new tserver with Master: 845d3ece65974e25ad6ce260ba27b693 (127.18.45.194:43321)
I20260806 22:45:10.023579 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.195:44409
--local_ip_for_outbound_sockets=127.18.45.195
--tserver_master_addrs=127.18.45.254:39045
--webserver_port=42419
--webserver_interface=127.18.45.195
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/krb5kdc/oryx_127.18.45.195.keytab
--principal=oryx/127.18.45.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:33219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.18.45.254:39045,127.18.45.253:40509
--tserver_master_addrs=127.18.45.254:39045,127.18.45.253:40509,127.18.45.252:40869 with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:45:10.071593 28802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:10.068443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52764 (local address 127.18.45.252:40869)
0806 22:45:10.068534 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:10.068537 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:10.068549 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:45:10.068590 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:10.068592 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:10.068633 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:10.068717 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:45:10.068907 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:10.069564 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:10.070625 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:10.071170 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:10.071259 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:10.071364 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:10.071429 (+ 65us) server_negotiation.cc:299] Negotiation successful
0806 22:45:10.071489 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 22:45:10.071593 28484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:10.068295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.252:40869 (local address 127.0.0.1:52764)
0806 22:45:10.068388 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:10.068403 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:10.068501 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:10.068724 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:10.068725 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:10.068728 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:45:10.068822 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:10.068826 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:10.069667 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:10.069669 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:10.070515 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:10.070518 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:10.071257 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:10.071258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:10.071354 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:10.071359 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:45:10.071417 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 22:45:10.071488 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260806 22:45:10.072042 28555 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260806 22:45:10.072239 28597 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc26ea2c399a41b3857e7d7f2a8baef6. Latest consensus state: current_term: 1 leader_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39045 } } peers { permanent_uuid: "8f1bacf8b0b7450f8cff03a77e4c0890" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 40509 } attrs { promote: false } } peers { permanent_uuid: "69b1737c4ee248689c79e5644bc1274d" member_type: NON_VOTER last_known_addr { host: "127.18.45.252" port: 40869 } attrs { promote: true } } }
I20260806 22:45:10.072400 28597 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d [sys.catalog]: This master's current role is: LEARNER
W20260806 22:45:10.125953 29007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:45:10.126277 29007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:45:10.126322 29007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:45:10.127794 29007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:45:10.127864 29007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:45:10.127903 29007 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.195
I20260806 22:45:10.129618 29007 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:33219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/krb5kdc/oryx_127.18.45.195.keytab
--principal=oryx/127.18.45.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.195:44409
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.45.195
--webserver_port=42419
--tserver_master_addrs=127.18.45.254:39045,127.18.45.253:40509,127.18.45.252:40869
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29007
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.195
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:45:10.129910 29007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:45:10.130224 29007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 22:45:10.136686 29007 server_base.cc:1034] running on GCE node
W20260806 22:45:10.136680 29018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:10.136653 29013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:10.136667 29012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:10.137326 29007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:45:10.137591 29007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:45:10.138793 29007 hybrid_clock.cc:648] HybridClock initialized: now 1786056310138780 us; error 38 us; skew 500 ppm
Aug 06 22:45:10 dist-test-slave-h3ft krb5kdc[27365](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056310, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.45.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:45:10.142213 29007 init.cc:377] Logged in from keytab as oryx/127.18.45.195@KRBTEST.COM (short username oryx)
I20260806 22:45:10.142802 29007 webserver.cc:460] Webserver started at http://127.18.45.195:42419/ using document root <none> and password file <none>
I20260806 22:45:10.143159 29007 fs_manager.cc:359] Metadata directory not provided
I20260806 22:45:10.143262 29007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:45:10.145126 29007 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 22:45:10.145916 29030 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:10.146247 29007 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:45:10.146368 29007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-2/wal
uuid: "06ca1eb660804c5f9d85b58de0a16cd2"
format_stamp: "Formatted at 2026-08-06 22:45:08 on dist-test-slave-h3ft"
server_key: "c55c1b23dc261ec2e7b9fea7fb5432a9"
server_key_iv: "87eae83df9dcc9c73a78e731619b180b"
server_key_version: "encryptionkey@0"
I20260806 22:45:10.146814 29007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:45:10.165611 29007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:45:10.168419 29007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:45:10.168615 29007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:45:10.168932 29007 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:45:10.169327 29007 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:45:10.169385 29007 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:10.169437 29007 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:45:10.169464 29007 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:10.176873 29007 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.195:44409
I20260806 22:45:10.176924 29196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.195:44409 every 8 connection(s)
I20260806 22:45:10.177350 29007 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
I20260806 22:45:10.181278 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 29007
Aug 06 22:45:10 dist-test-slave-h3ft krb5kdc[27365](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056310, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.45.195@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM
Aug 06 22:45:10 dist-test-slave-h3ft krb5kdc[27365](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056310, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.45.195@KRBTEST.COM for oryx/127.18.45.252@KRBTEST.COM
Aug 06 22:45:10 dist-test-slave-h3ft krb5kdc[27365](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056310, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.45.195@KRBTEST.COM for oryx/127.18.45.253@KRBTEST.COM
I20260806 22:45:10.193079 27504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:10.178614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.195:47789 (local address 127.18.45.254:39045)
0806 22:45:10.178698 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:10.178701 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:10.178720 (+ 19us) server_negotiation.cc:407] Connection header received
0806 22:45:10.180082 (+ 1362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:10.180084 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:10.180101 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:10.180140 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:10.181121 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:10.181990 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:10.183739 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:10.184048 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:10.184095 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:10.189361 (+ 5266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:10.189362 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:10.189363 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:10.189371 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:10.191231 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:10.191799 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:10.191800 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:10.191801 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:10.191868 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:10.192190 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:10.192192 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:10.192192 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:10.192365 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:10.192430 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:10.192871 (+ 441us) server_negotiation.cc:299] Negotiation successful
0806 22:45:10.192971 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 22:45:10.194761 29209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:10.178273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.253:40509 (local address 127.18.45.195:34489)
0806 22:45:10.178818 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:10.178830 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:10.179937 (+ 1107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:10.180245 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:10.180250 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:10.180720 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:10.180997 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:10.181002 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:10.183303 (+ 2301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:10.183304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:10.184175 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:10.184179 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:10.184706 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:10.184707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:10.184779 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:10.185255 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:10.185261 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:10.188760 (+ 3499us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:45:10.192995 (+ 4235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:10.192996 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:10.192996 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:10.193558 (+ 562us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:10.193781 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:10.193782 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:10.193783 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:10.193852 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:10.194208 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:45:10.194209 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:45:10.194255 (+ 46us) client_negotiation.cc:769] Sending connection context
0806 22:45:10.194322 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 22:45:10.194461 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"mutex_wait_us":389,"thread_start_us":88,"threads_started":1}
I20260806 22:45:10.195067 28802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:10.178355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.195:45669 (local address 127.18.45.252:40869)
0806 22:45:10.178503 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:10.178506 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:10.178701 (+ 195us) server_negotiation.cc:407] Connection header received
0806 22:45:10.180080 (+ 1379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:10.180082 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:10.180104 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:10.180140 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:10.181121 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:10.181990 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:10.183739 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:10.184065 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:10.184103 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:10.190694 (+ 6591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:10.190695 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:10.190695 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:10.190703 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:10.192905 (+ 2202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:10.194085 (+ 1180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:10.194087 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:10.194088 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:10.194161 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:10.194393 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:10.194394 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:10.194395 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:10.194539 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:10.194598 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:10.194806 (+ 208us) server_negotiation.cc:299] Negotiation successful
0806 22:45:10.194905 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115}
I20260806 22:45:10.195401 29206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:10.178273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.252:40869 (local address 127.18.45.195:45669)
0806 22:45:10.178503 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:10.178557 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:10.179933 (+ 1376us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:10.180247 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:10.180251 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:10.180741 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:10.180998 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:10.181005 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:10.182166 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:10.182167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:10.183631 (+ 1464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:10.183634 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:10.184206 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:10.184206 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:10.184360 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:10.185120 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:10.185133 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:10.189022 (+ 3889us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:45:10.193166 (+ 4144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:10.193167 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:10.193168 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:10.193976 (+ 808us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:10.194245 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:10.194245 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:10.194246 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:10.194313 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:10.194635 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:45:10.194636 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:45:10.194684 (+ 48us) client_negotiation.cc:769] Sending connection context
0806 22:45:10.195155 (+ 471us) client_negotiation.cc:240] Negotiation successful
0806 22:45:10.195287 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":60,"thread_start_us":114,"threads_started":1}
I20260806 22:45:10.195871 29197 heartbeater.cc:347] Connected to a master server at 127.18.45.252:40869
I20260806 22:45:10.195914 28342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:10.178439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.195:34489 (local address 127.18.45.253:40509)
0806 22:45:10.178571 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:10.178574 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:10.178916 (+ 342us) server_negotiation.cc:407] Connection header received
0806 22:45:10.180066 (+ 1150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:10.180067 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:10.180087 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:10.180146 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:10.182412 (+ 2266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:10.183197 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:10.184276 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:10.184568 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:10.184636 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:10.188868 (+ 4232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:10.188869 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:10.188870 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:10.188878 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:10.192013 (+ 3135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:10.193633 (+ 1620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:10.193635 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:10.193635 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:10.193702 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:10.193942 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:10.193943 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:10.193944 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:10.194113 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:10.194181 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:10.195728 (+ 1547us) server_negotiation.cc:299] Negotiation successful
0806 22:45:10.195814 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260806 22:45:10.195925 29225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:10.182973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56064 (local address 127.18.45.254:39045)
0806 22:45:10.186499 (+ 3526us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:10.186500 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:10.186507 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:45:10.186526 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:10.186527 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:10.186538 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:10.186632 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:10.187475 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:10.188045 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:10.189072 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:10.189398 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:10.189461 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:10.191133 (+ 1672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:10.191135 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:10.191135 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:10.191144 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:10.193338 (+ 2194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:10.194706 (+ 1368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:10.194708 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:10.194708 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:10.194774 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:10.195062 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:10.195063 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:10.195064 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:10.195233 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:10.195336 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:10.195706 (+ 370us) server_negotiation.cc:299] Negotiation successful
0806 22:45:10.195805 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3503,"thread_start_us":86,"threads_started":1}
I20260806 22:45:10.196008 29197 heartbeater.cc:464] Registering TS with master...
I20260806 22:45:10.196241 29204 heartbeater.cc:347] Connected to a master server at 127.18.45.253:40509
I20260806 22:45:10.196281 29197 heartbeater.cc:511] Master 127.18.45.252:40869 requested a full tablet report, sending...
I20260806 22:45:10.196282 29204 heartbeater.cc:464] Registering TS with master...
I20260806 22:45:10.196382 29204 heartbeater.cc:511] Master 127.18.45.253:40509 requested a full tablet report, sending...
I20260806 22:45:10.196846 29207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:10.178273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39045 (local address 127.18.45.195:47789)
0806 22:45:10.178663 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:10.178673 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:10.179933 (+ 1260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:10.180249 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:10.180251 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:10.180741 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:10.180998 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:10.181005 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:10.182166 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:10.182167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:10.183631 (+ 1464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:10.183634 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:10.184203 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:10.184204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:10.184360 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:10.185402 (+ 1042us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:10.185407 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:10.189258 (+ 3851us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:45:10.191319 (+ 2061us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:10.191327 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:10.191331 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:10.191699 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:10.191944 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:10.191945 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:10.191946 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:10.192097 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:10.192470 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:45:10.192478 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:45:10.192694 (+ 216us) client_negotiation.cc:769] Sending connection context
0806 22:45:10.192956 (+ 262us) client_negotiation.cc:240] Negotiation successful
0806 22:45:10.193674 (+ 718us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"mutex_wait_us":122,"thread_start_us":111,"threads_started":1}
I20260806 22:45:10.196883 28539 ts_manager.cc:194] Registered new tserver with Master: 06ca1eb660804c5f9d85b58de0a16cd2 (127.18.45.195:44409)
I20260806 22:45:10.197319 28274 ts_manager.cc:194] Registered new tserver with Master: 06ca1eb660804c5f9d85b58de0a16cd2 (127.18.45.195:44409)
I20260806 22:45:10.197336 29200 heartbeater.cc:347] Connected to a master server at 127.18.45.254:39045
I20260806 22:45:10.197391 29200 heartbeater.cc:464] Registering TS with master...
I20260806 22:45:10.197461 29200 heartbeater.cc:511] Master 127.18.45.254:39045 requested a full tablet report, sending...
I20260806 22:45:10.197696 27431 ts_manager.cc:194] Re-registered known tserver with Master: 06ca1eb660804c5f9d85b58de0a16cd2 (127.18.45.195:44409)
I20260806 22:45:10.198197 27431 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.45.195@KRBTEST.COM'} at 127.18.45.195:47789
I20260806 22:45:10.198383 29227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:10.183460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52780 (local address 127.18.45.252:40869)
0806 22:45:10.184036 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:10.184038 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:10.184181 (+ 143us) server_negotiation.cc:407] Connection header received
0806 22:45:10.184206 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:10.184207 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:10.184224 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:10.184287 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:10.185283 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:10.188684 (+ 3401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:10.190144 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:10.190508 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:10.190588 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:10.192926 (+ 2338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:10.192927 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:10.192928 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:10.192936 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:10.195109 (+ 2173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:10.195586 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:10.195587 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:10.195587 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:10.195783 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:10.197310 (+ 1527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:10.197311 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:10.197312 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:10.197475 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:10.197585 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:10.198095 (+ 510us) server_negotiation.cc:299] Negotiation successful
0806 22:45:10.198193 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":69,"threads_started":1}
I20260806 22:45:10.200238 29226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:10.183187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55804 (local address 127.18.45.253:40509)
0806 22:45:10.184996 (+ 1809us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:10.184998 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:10.185006 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:45:10.185029 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:10.185030 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:10.185058 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:10.186220 (+ 1162us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:10.186270 (+ 50us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:10.187029 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:10.188194 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:10.188488 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:10.188554 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:10.194400 (+ 5846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:10.194401 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:10.194402 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:10.194408 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:10.199150 (+ 4742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:10.199511 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:10.199512 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:10.199512 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:10.199573 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:10.199769 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:10.199770 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:10.199770 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:10.199877 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:10.199954 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:10.200088 (+ 134us) server_negotiation.cc:299] Negotiation successful
0806 22:45:10.200170 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1784,"thread_start_us":80,"threads_started":1}
I20260806 22:45:10.199255 27431 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56064:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:45:10.201815 27431 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:45:10.202714 28489 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc26ea2c399a41b3857e7d7f2a8baef6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39045 } } peers { permanent_uuid: "8f1bacf8b0b7450f8cff03a77e4c0890" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 40509 } attrs { promote: false } } peers { permanent_uuid: "69b1737c4ee248689c79e5644bc1274d" member_type: NON_VOTER last_known_addr { host: "127.18.45.252" port: 40869 } attrs { promote: true } } }
I20260806 22:45:10.202833 28489 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc26ea2c399a41b3857e7d7f2a8baef6 [sys.catalog]: This master's current role is: LEADER
I20260806 22:45:10.204382 28489 raft_consensus.cc:1050] T 00000000000000000000000000000000 P bc26ea2c399a41b3857e7d7f2a8baef6: attempting to promote NON_VOTER 69b1737c4ee248689c79e5644bc1274d to VOTER
I20260806 22:45:10.204524 28489 consensus_queue.cc:237] T 00000000000000000000000000000000 P bc26ea2c399a41b3857e7d7f2a8baef6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39045 } } peers { permanent_uuid: "8f1bacf8b0b7450f8cff03a77e4c0890" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 40509 } attrs { promote: false } } peers { permanent_uuid: "69b1737c4ee248689c79e5644bc1274d" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 40869 } attrs { promote: false } }
I20260806 22:45:10.204919 28555 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d [term 1 LEARNER]: Refusing update from remote peer bc26ea2c399a41b3857e7d7f2a8baef6: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260806 22:45:10.205128 29230 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bc26ea2c399a41b3857e7d7f2a8baef6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "69b1737c4ee248689c79e5644bc1274d" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 40869 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260806 22:45:10.205579 28289 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890 [term 1 FOLLOWER]: Refusing update from remote peer bc26ea2c399a41b3857e7d7f2a8baef6: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260806 22:45:10.205904 29230 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bc26ea2c399a41b3857e7d7f2a8baef6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f1bacf8b0b7450f8cff03a77e4c0890" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 40509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260806 22:45:10.206218 29230 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bc26ea2c399a41b3857e7d7f2a8baef6 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 69b1737c4ee248689c79e5644bc1274d (127.18.45.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39045 } } peers { permanent_uuid: "8f1bacf8b0b7450f8cff03a77e4c0890" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 40509 } attrs { promote: false } } peers { permanent_uuid: "69b1737c4ee248689c79e5644bc1274d" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 40869 } attrs { promote: false } } }
I20260806 22:45:10.206172 28555 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 69b1737c4ee248689c79e5644bc1274d (127.18.45.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39045 } } peers { permanent_uuid: "8f1bacf8b0b7450f8cff03a77e4c0890" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 40509 } attrs { promote: false } } peers { permanent_uuid: "69b1737c4ee248689c79e5644bc1274d" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 40869 } attrs { promote: false } } }
I20260806 22:45:10.206642 28289 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 69b1737c4ee248689c79e5644bc1274d (127.18.45.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39045 } } peers { permanent_uuid: "8f1bacf8b0b7450f8cff03a77e4c0890" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 40509 } attrs { promote: false } } peers { permanent_uuid: "69b1737c4ee248689c79e5644bc1274d" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 40869 } attrs { promote: false } } }
I20260806 22:45:10.207492 29230 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc26ea2c399a41b3857e7d7f2a8baef6 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39045 } } peers { permanent_uuid: "8f1bacf8b0b7450f8cff03a77e4c0890" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 40509 } attrs { promote: false } } peers { permanent_uuid: "69b1737c4ee248689c79e5644bc1274d" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 40869 } attrs { promote: false } } }
I20260806 22:45:10.207592 29230 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc26ea2c399a41b3857e7d7f2a8baef6 [sys.catalog]: This master's current role is: LEADER
I20260806 22:45:10.207737 29230 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc26ea2c399a41b3857e7d7f2a8baef6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39045 } } peers { permanent_uuid: "8f1bacf8b0b7450f8cff03a77e4c0890" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 40509 } attrs { promote: false } } peers { permanent_uuid: "69b1737c4ee248689c79e5644bc1274d" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 40869 } attrs { promote: false } } }
I20260806 22:45:10.207744 28489 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc26ea2c399a41b3857e7d7f2a8baef6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39045 } } peers { permanent_uuid: "8f1bacf8b0b7450f8cff03a77e4c0890" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 40509 } attrs { promote: false } } peers { permanent_uuid: "69b1737c4ee248689c79e5644bc1274d" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 40869 } attrs { promote: false } } }
I20260806 22:45:10.207818 29230 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc26ea2c399a41b3857e7d7f2a8baef6 [sys.catalog]: This master's current role is: LEADER
I20260806 22:45:10.207828 28489 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc26ea2c399a41b3857e7d7f2a8baef6 [sys.catalog]: This master's current role is: LEADER
I20260806 22:45:10.207890 28597 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39045 } } peers { permanent_uuid: "8f1bacf8b0b7450f8cff03a77e4c0890" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 40509 } attrs { promote: false } } peers { permanent_uuid: "69b1737c4ee248689c79e5644bc1274d" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 40869 } attrs { promote: false } } }
I20260806 22:45:10.207970 28597 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:45:10.208253 29247 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39045 } } peers { permanent_uuid: "8f1bacf8b0b7450f8cff03a77e4c0890" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 40509 } attrs { promote: false } } peers { permanent_uuid: "69b1737c4ee248689c79e5644bc1274d" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 40869 } attrs { promote: false } } }
I20260806 22:45:10.208375 29247 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:45:10.213359 29250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:10.209264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.193:37419 (local address 127.0.0.1:45258)
0806 22:45:10.209562 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:10.209572 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:10.209618 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:10.209984 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:10.209985 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:10.209988 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:45:10.210058 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:10.210061 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:10.211241 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:10.211242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:10.212003 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:10.212005 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:10.213079 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:10.213080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:10.213146 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:10.213152 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 22:45:10.213221 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 22:45:10.213270 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":139,"threads_started":1}
I20260806 22:45:10.213392 29252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:10.209567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45258 (local address 127.18.45.193:37419)
0806 22:45:10.209780 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:10.209785 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:10.209797 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:45:10.209850 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:10.209853 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:10.209873 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:10.209991 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:45:10.210147 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:10.211133 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:10.212099 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:10.212978 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:10.213057 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:10.213154 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:10.213238 (+ 84us) server_negotiation.cc:299] Negotiation successful
0806 22:45:10.213279 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":129,"threads_started":1}
I20260806 22:45:10.214421 28484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:10.209162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.194:43321 (local address 127.0.0.1:47092)
0806 22:45:10.209203 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:10.209217 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:10.209263 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:10.210258 (+ 995us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:10.210258 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:10.210260 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:45:10.210297 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:10.210298 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:10.212225 (+ 1927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:10.212226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:10.213312 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:10.213315 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:10.214168 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:10.214168 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:10.214233 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:10.214238 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:45:10.214297 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 22:45:10.214352 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20,"mutex_wait_us":24}
I20260806 22:45:10.214463 29251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:10.209322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47092 (local address 127.18.45.194:43321)
0806 22:45:10.209773 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:10.209780 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:10.209794 (+ 14us) server_negotiation.cc:407] Connection header received
0806 22:45:10.210138 (+ 344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:10.210141 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:10.210161 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:10.210248 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:45:10.210378 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:10.212087 (+ 1709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:10.213425 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:10.214072 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:10.214165 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:10.214255 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:10.214314 (+ 59us) server_negotiation.cc:299] Negotiation successful
0806 22:45:10.214367 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":100,"threads_started":1}
I20260806 22:45:10.214293 28699 tablet_service.cc:1467] Processing CreateTablet for tablet 08130e558b53463daf675faed00da604 (DEFAULT_TABLE table=test-table [id=099b230927ce451f8682297fdc24dbea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:45:10.214639 29248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:10.209263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.195:44409 (local address 127.0.0.1:34192)
0806 22:45:10.209409 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:10.209419 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:10.209471 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:10.210039 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:10.210040 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:10.210043 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:45:10.210109 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:10.210111 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:10.212269 (+ 2158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:10.212270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:10.213146 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:10.213148 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:10.214421 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:10.214423 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:10.214486 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:10.214490 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:45:10.214523 (+ 33us) client_negotiation.cc:240] Negotiation successful
0806 22:45:10.214568 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":108,"threads_started":1}
I20260806 22:45:10.214833 28699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08130e558b53463daf675faed00da604. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:45:10.215535 29255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:10.209550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34192 (local address 127.18.45.195:44409)
0806 22:45:10.209846 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:10.209850 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:10.209859 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:45:10.209903 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:10.209906 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:10.209925 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:10.210203 (+ 278us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:45:10.210278 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:10.212169 (+ 1891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:10.213256 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:10.214315 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:10.215246 (+ 931us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:10.215355 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:10.215386 (+ 31us) server_negotiation.cc:299] Negotiation successful
0806 22:45:10.215442 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":134,"threads_started":1}
I20260806 22:45:10.216289 29104 tablet_service.cc:1467] Processing CreateTablet for tablet 08130e558b53463daf675faed00da604 (DEFAULT_TABLE table=test-table [id=099b230927ce451f8682297fdc24dbea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:45:10.216722 29104 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08130e558b53463daf675faed00da604. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:45:10.218580 29256 tablet_bootstrap.cc:492] T 08130e558b53463daf675faed00da604 P 0bc1abeccf9a48f7b80fcde36b6c6a1e: Bootstrap starting.
I20260806 22:45:10.219158 28898 tablet_service.cc:1467] Processing CreateTablet for tablet 08130e558b53463daf675faed00da604 (DEFAULT_TABLE table=test-table [id=099b230927ce451f8682297fdc24dbea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:45:10.219575 29256 tablet_bootstrap.cc:654] T 08130e558b53463daf675faed00da604 P 0bc1abeccf9a48f7b80fcde36b6c6a1e: Neither blocks nor log segments found. Creating new log.
I20260806 22:45:10.219619 28898 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08130e558b53463daf675faed00da604. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:45:10.219990 29256 log.cc:824] T 08130e558b53463daf675faed00da604 P 0bc1abeccf9a48f7b80fcde36b6c6a1e: Log is configured to *not* fsync() on all Append() calls
I20260806 22:45:10.220175 29257 tablet_bootstrap.cc:492] T 08130e558b53463daf675faed00da604 P 06ca1eb660804c5f9d85b58de0a16cd2: Bootstrap starting.
I20260806 22:45:10.220933 29257 tablet_bootstrap.cc:654] T 08130e558b53463daf675faed00da604 P 06ca1eb660804c5f9d85b58de0a16cd2: Neither blocks nor log segments found. Creating new log.
I20260806 22:45:10.221125 29256 tablet_bootstrap.cc:492] T 08130e558b53463daf675faed00da604 P 0bc1abeccf9a48f7b80fcde36b6c6a1e: No bootstrap required, opened a new log
I20260806 22:45:10.221215 29256 ts_tablet_manager.cc:1397] T 08130e558b53463daf675faed00da604 P 0bc1abeccf9a48f7b80fcde36b6c6a1e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:45:10.221328 29257 log.cc:824] T 08130e558b53463daf675faed00da604 P 06ca1eb660804c5f9d85b58de0a16cd2: Log is configured to *not* fsync() on all Append() calls
I20260806 22:45:10.222060 29257 tablet_bootstrap.cc:492] T 08130e558b53463daf675faed00da604 P 06ca1eb660804c5f9d85b58de0a16cd2: No bootstrap required, opened a new log
I20260806 22:45:10.222158 29257 ts_tablet_manager.cc:1397] T 08130e558b53463daf675faed00da604 P 06ca1eb660804c5f9d85b58de0a16cd2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:45:10.224054 29261 tablet_bootstrap.cc:492] T 08130e558b53463daf675faed00da604 P 845d3ece65974e25ad6ce260ba27b693: Bootstrap starting.
I20260806 22:45:10.224381 29257 raft_consensus.cc:348] T 08130e558b53463daf675faed00da604 P 06ca1eb660804c5f9d85b58de0a16cd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "845d3ece65974e25ad6ce260ba27b693" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 43321 } } peers { permanent_uuid: "06ca1eb660804c5f9d85b58de0a16cd2" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44409 } } peers { permanent_uuid: "0bc1abeccf9a48f7b80fcde36b6c6a1e" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 37419 } }
I20260806 22:45:10.224606 29257 raft_consensus.cc:374] T 08130e558b53463daf675faed00da604 P 06ca1eb660804c5f9d85b58de0a16cd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:45:10.224643 29257 raft_consensus.cc:729] T 08130e558b53463daf675faed00da604 P 06ca1eb660804c5f9d85b58de0a16cd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06ca1eb660804c5f9d85b58de0a16cd2, State: Initialized, Role: FOLLOWER
I20260806 22:45:10.224555 29256 raft_consensus.cc:348] T 08130e558b53463daf675faed00da604 P 0bc1abeccf9a48f7b80fcde36b6c6a1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "845d3ece65974e25ad6ce260ba27b693" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 43321 } } peers { permanent_uuid: "06ca1eb660804c5f9d85b58de0a16cd2" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44409 } } peers { permanent_uuid: "0bc1abeccf9a48f7b80fcde36b6c6a1e" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 37419 } }
I20260806 22:45:10.224736 29256 raft_consensus.cc:374] T 08130e558b53463daf675faed00da604 P 0bc1abeccf9a48f7b80fcde36b6c6a1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:45:10.224781 29256 raft_consensus.cc:729] T 08130e558b53463daf675faed00da604 P 0bc1abeccf9a48f7b80fcde36b6c6a1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bc1abeccf9a48f7b80fcde36b6c6a1e, State: Initialized, Role: FOLLOWER
I20260806 22:45:10.224799 29257 consensus_queue.cc:260] T 08130e558b53463daf675faed00da604 P 06ca1eb660804c5f9d85b58de0a16cd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "845d3ece65974e25ad6ce260ba27b693" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 43321 } } peers { permanent_uuid: "06ca1eb660804c5f9d85b58de0a16cd2" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44409 } } peers { permanent_uuid: "0bc1abeccf9a48f7b80fcde36b6c6a1e" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 37419 } }
I20260806 22:45:10.224911 29256 consensus_queue.cc:260] T 08130e558b53463daf675faed00da604 P 0bc1abeccf9a48f7b80fcde36b6c6a1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "845d3ece65974e25ad6ce260ba27b693" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 43321 } } peers { permanent_uuid: "06ca1eb660804c5f9d85b58de0a16cd2" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44409 } } peers { permanent_uuid: "0bc1abeccf9a48f7b80fcde36b6c6a1e" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 37419 } }
I20260806 22:45:10.225026 29261 tablet_bootstrap.cc:654] T 08130e558b53463daf675faed00da604 P 845d3ece65974e25ad6ce260ba27b693: Neither blocks nor log segments found. Creating new log.
I20260806 22:45:10.225201 29256 ts_tablet_manager.cc:1428] T 08130e558b53463daf675faed00da604 P 0bc1abeccf9a48f7b80fcde36b6c6a1e: Time spent starting tablet: real 0.004s user 0.001s sys 0.003s
I20260806 22:45:10.225201 29257 ts_tablet_manager.cc:1428] T 08130e558b53463daf675faed00da604 P 06ca1eb660804c5f9d85b58de0a16cd2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:45:10.225303 28797 heartbeater.cc:503] Master 127.18.45.254:39045 was elected leader, sending a full tablet report...
I20260806 22:45:10.225697 29261 log.cc:824] T 08130e558b53463daf675faed00da604 P 845d3ece65974e25ad6ce260ba27b693: Log is configured to *not* fsync() on all Append() calls
I20260806 22:45:10.226639 29200 heartbeater.cc:503] Master 127.18.45.254:39045 was elected leader, sending a full tablet report...
I20260806 22:45:10.227919 29261 tablet_bootstrap.cc:492] T 08130e558b53463daf675faed00da604 P 845d3ece65974e25ad6ce260ba27b693: No bootstrap required, opened a new log
I20260806 22:45:10.228029 29261 ts_tablet_manager.cc:1397] T 08130e558b53463daf675faed00da604 P 845d3ece65974e25ad6ce260ba27b693: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 22:45:10.230389 29261 raft_consensus.cc:348] T 08130e558b53463daf675faed00da604 P 845d3ece65974e25ad6ce260ba27b693 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "845d3ece65974e25ad6ce260ba27b693" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 43321 } } peers { permanent_uuid: "06ca1eb660804c5f9d85b58de0a16cd2" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44409 } } peers { permanent_uuid: "0bc1abeccf9a48f7b80fcde36b6c6a1e" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 37419 } }
I20260806 22:45:10.230568 29261 raft_consensus.cc:374] T 08130e558b53463daf675faed00da604 P 845d3ece65974e25ad6ce260ba27b693 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:45:10.230592 29261 raft_consensus.cc:729] T 08130e558b53463daf675faed00da604 P 845d3ece65974e25ad6ce260ba27b693 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 845d3ece65974e25ad6ce260ba27b693, State: Initialized, Role: FOLLOWER
I20260806 22:45:10.230733 29261 consensus_queue.cc:260] T 08130e558b53463daf675faed00da604 P 845d3ece65974e25ad6ce260ba27b693 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "845d3ece65974e25ad6ce260ba27b693" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 43321 } } peers { permanent_uuid: "06ca1eb660804c5f9d85b58de0a16cd2" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44409 } } peers { permanent_uuid: "0bc1abeccf9a48f7b80fcde36b6c6a1e" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 37419 } }
I20260806 22:45:10.231101 28999 heartbeater.cc:503] Master 127.18.45.254:39045 was elected leader, sending a full tablet report...
I20260806 22:45:10.231081 29261 ts_tablet_manager.cc:1428] T 08130e558b53463daf675faed00da604 P 845d3ece65974e25ad6ce260ba27b693: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
W20260806 22:45:10.234144 29002 tablet.cc:2369] T 08130e558b53463daf675faed00da604 P 845d3ece65974e25ad6ce260ba27b693: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:45:10.316452 28799 tablet.cc:2369] T 08130e558b53463daf675faed00da604 P 0bc1abeccf9a48f7b80fcde36b6c6a1e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:45:10.332788 29266 raft_consensus.cc:482] T 08130e558b53463daf675faed00da604 P 845d3ece65974e25ad6ce260ba27b693 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:45:10.332975 29266 raft_consensus.cc:504] T 08130e558b53463daf675faed00da604 P 845d3ece65974e25ad6ce260ba27b693 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "845d3ece65974e25ad6ce260ba27b693" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 43321 } } peers { permanent_uuid: "06ca1eb660804c5f9d85b58de0a16cd2" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44409 } } peers { permanent_uuid: "0bc1abeccf9a48f7b80fcde36b6c6a1e" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 37419 } }
I20260806 22:45:10.333520 29266 leader_election.cc:288] T 08130e558b53463daf675faed00da604 P 845d3ece65974e25ad6ce260ba27b693 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06ca1eb660804c5f9d85b58de0a16cd2 (127.18.45.195:44409), 0bc1abeccf9a48f7b80fcde36b6c6a1e (127.18.45.193:37419)
I20260806 22:45:10.336743 29255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:10.333934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:59873 (local address 127.18.45.195:44409)
0806 22:45:10.333997 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:10.333999 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:10.334009 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:45:10.334062 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:10.334063 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:10.334072 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:10.334150 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:45:10.334365 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:10.334859 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:10.335943 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:10.336329 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:10.336390 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:10.336400 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:10.336586 (+ 186us) server_negotiation.cc:299] Negotiation successful
0806 22:45:10.336648 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 22:45:10.336761 29252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:10.333888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:45029 (local address 127.18.45.193:37419)
0806 22:45:10.333943 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:10.333945 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:10.334008 (+ 63us) server_negotiation.cc:407] Connection header received
0806 22:45:10.334045 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:10.334046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:10.334061 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:10.334112 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:45:10.334323 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:10.334859 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:10.335940 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:10.336374 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:10.336452 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:10.336462 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:10.336635 (+ 173us) server_negotiation.cc:299] Negotiation successful
0806 22:45:10.336680 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 22:45:10.336761 29005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:10.333841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.195:44409 (local address 127.18.45.194:59873)
0806 22:45:10.333905 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:10.333917 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:10.333968 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:10.334171 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:10.334172 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:10.334174 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:45:10.334267 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:10.334268 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:10.334956 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:10.334957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:10.335843 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:10.335845 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:10.336414 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:10.336415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:10.336513 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:10.336517 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:45:10.336562 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 22:45:10.336628 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":39}
I20260806 22:45:10.336774 29003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:10.333809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.193:37419 (local address 127.18.45.194:45029)
0806 22:45:10.333889 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:10.333905 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:10.333960 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:10.334147 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:10.334148 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:10.334150 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:45:10.334251 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:10.334253 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:10.334952 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:10.334953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:10.335845 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:10.335847 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:10.336462 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:10.336463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:10.336539 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:10.336542 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:45:10.336618 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 22:45:10.336669 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52,"mutex_wait_us":2}
I20260806 22:45:10.337235 28728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08130e558b53463daf675faed00da604" candidate_uuid: "845d3ece65974e25ad6ce260ba27b693" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bc1abeccf9a48f7b80fcde36b6c6a1e" is_pre_election: true
I20260806 22:45:10.337481 28728 raft_consensus.cc:2454] T 08130e558b53463daf675faed00da604 P 0bc1abeccf9a48f7b80fcde36b6c6a1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 845d3ece65974e25ad6ce260ba27b693 in term 0.
I20260806 22:45:10.337687 29131 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08130e558b53463daf675faed00da604" candidate_uuid: "845d3ece65974e25ad6ce260ba27b693" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ca1eb660804c5f9d85b58de0a16cd2" is_pre_election: true
I20260806 22:45:10.337837 28829 leader_election.cc:302] T 08130e558b53463daf675faed00da604 P 845d3ece65974e25ad6ce260ba27b693 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bc1abeccf9a48f7b80fcde36b6c6a1e, 845d3ece65974e25ad6ce260ba27b693; no voters:
I20260806 22:45:10.337893 29131 raft_consensus.cc:2454] T 08130e558b53463daf675faed00da604 P 06ca1eb660804c5f9d85b58de0a16cd2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 845d3ece65974e25ad6ce260ba27b693 in term 0.
I20260806 22:45:10.338066 29266 raft_consensus.cc:2781] T 08130e558b53463daf675faed00da604 P 845d3ece65974e25ad6ce260ba27b693 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:45:10.338137 29266 raft_consensus.cc:482] T 08130e558b53463daf675faed00da604 P 845d3ece65974e25ad6ce260ba27b693 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:45:10.338163 29266 raft_consensus.cc:3037] T 08130e558b53463daf675faed00da604 P 845d3ece65974e25ad6ce260ba27b693 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:45:10.339193 29266 raft_consensus.cc:504] T 08130e558b53463daf675faed00da604 P 845d3ece65974e25ad6ce260ba27b693 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "845d3ece65974e25ad6ce260ba27b693" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 43321 } } peers { permanent_uuid: "06ca1eb660804c5f9d85b58de0a16cd2" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44409 } } peers { permanent_uuid: "0bc1abeccf9a48f7b80fcde36b6c6a1e" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 37419 } }
I20260806 22:45:10.339395 29266 leader_election.cc:288] T 08130e558b53463daf675faed00da604 P 845d3ece65974e25ad6ce260ba27b693 [CANDIDATE]: Term 1 election: Requested vote from peers 06ca1eb660804c5f9d85b58de0a16cd2 (127.18.45.195:44409), 0bc1abeccf9a48f7b80fcde36b6c6a1e (127.18.45.193:37419)
I20260806 22:45:10.339586 28728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08130e558b53463daf675faed00da604" candidate_uuid: "845d3ece65974e25ad6ce260ba27b693" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bc1abeccf9a48f7b80fcde36b6c6a1e"
I20260806 22:45:10.339560 29131 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08130e558b53463daf675faed00da604" candidate_uuid: "845d3ece65974e25ad6ce260ba27b693" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06ca1eb660804c5f9d85b58de0a16cd2"
I20260806 22:45:10.339684 29131 raft_consensus.cc:3037] T 08130e558b53463daf675faed00da604 P 06ca1eb660804c5f9d85b58de0a16cd2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:45:10.339684 28728 raft_consensus.cc:3037] T 08130e558b53463daf675faed00da604 P 0bc1abeccf9a48f7b80fcde36b6c6a1e [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:45:10.340754 28728 raft_consensus.cc:2454] T 08130e558b53463daf675faed00da604 P 0bc1abeccf9a48f7b80fcde36b6c6a1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 845d3ece65974e25ad6ce260ba27b693 in term 1.
I20260806 22:45:10.340754 29131 raft_consensus.cc:2454] T 08130e558b53463daf675faed00da604 P 06ca1eb660804c5f9d85b58de0a16cd2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 845d3ece65974e25ad6ce260ba27b693 in term 1.
I20260806 22:45:10.340934 28829 leader_election.cc:302] T 08130e558b53463daf675faed00da604 P 845d3ece65974e25ad6ce260ba27b693 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bc1abeccf9a48f7b80fcde36b6c6a1e, 845d3ece65974e25ad6ce260ba27b693; no voters:
I20260806 22:45:10.341094 29266 raft_consensus.cc:2781] T 08130e558b53463daf675faed00da604 P 845d3ece65974e25ad6ce260ba27b693 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:45:10.341281 29266 raft_consensus.cc:686] T 08130e558b53463daf675faed00da604 P 845d3ece65974e25ad6ce260ba27b693 [term 1 LEADER]: Becoming Leader. State: Replica: 845d3ece65974e25ad6ce260ba27b693, State: Running, Role: LEADER
I20260806 22:45:10.341516 29266 consensus_queue.cc:237] T 08130e558b53463daf675faed00da604 P 845d3ece65974e25ad6ce260ba27b693 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "845d3ece65974e25ad6ce260ba27b693" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 43321 } } peers { permanent_uuid: "06ca1eb660804c5f9d85b58de0a16cd2" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44409 } } peers { permanent_uuid: "0bc1abeccf9a48f7b80fcde36b6c6a1e" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 37419 } }
I20260806 22:45:10.342669 27431 catalog_manager.cc:5495] T 08130e558b53463daf675faed00da604 P 845d3ece65974e25ad6ce260ba27b693 reported cstate change: term changed from 0 to 1, leader changed from <none> to 845d3ece65974e25ad6ce260ba27b693 (127.18.45.194). New cstate: current_term: 1 leader_uuid: "845d3ece65974e25ad6ce260ba27b693" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "845d3ece65974e25ad6ce260ba27b693" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 43321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06ca1eb660804c5f9d85b58de0a16cd2" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bc1abeccf9a48f7b80fcde36b6c6a1e" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 37419 } health_report { overall_health: UNKNOWN } } }
I20260806 22:45:10.363890 29251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:10.361311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47094 (local address 127.18.45.194:43321)
0806 22:45:10.361361 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:10.361363 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:10.361374 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:45:10.361459 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:10.361459 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:10.361483 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:10.361558 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:45:10.361730 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:10.362250 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:10.363078 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:10.363278 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:10.363335 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:10.363539 (+ 204us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:45:10.363663 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:45:10.363732 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:10.363793 (+ 61us) server_negotiation.cc:299] Negotiation successful
0806 22:45:10.363814 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 22:45:10.366310 29131 raft_consensus.cc:1261] T 08130e558b53463daf675faed00da604 P 06ca1eb660804c5f9d85b58de0a16cd2 [term 1 FOLLOWER]: Refusing update from remote peer 845d3ece65974e25ad6ce260ba27b693: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:45:10.366310 28728 raft_consensus.cc:1261] T 08130e558b53463daf675faed00da604 P 0bc1abeccf9a48f7b80fcde36b6c6a1e [term 1 FOLLOWER]: Refusing update from remote peer 845d3ece65974e25ad6ce260ba27b693: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:45:10.366783 29266 consensus_queue.cc:1035] T 08130e558b53463daf675faed00da604 P 845d3ece65974e25ad6ce260ba27b693 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06ca1eb660804c5f9d85b58de0a16cd2" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:45:10.366986 29270 consensus_queue.cc:1035] T 08130e558b53463daf675faed00da604 P 845d3ece65974e25ad6ce260ba27b693 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bc1abeccf9a48f7b80fcde36b6c6a1e" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 37419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:45:10.370261 29274 mvcc.cc:205] Tried to move back new op lower bound from 7315686647256940544 to 7315686647159635968. Current Snapshot: MvccSnapshot[applied={T|T < 7315686647256940544}]
I20260806 22:45:10.374121 27430 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:45:10.374255 27430 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:45:10.376633 27430 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bc26ea2c399a41b3857e7d7f2a8baef6: Sending DeleteTablet for 3 replicas of tablet 08130e558b53463daf675faed00da604
I20260806 22:45:10.377125 28699 tablet_service.cc:1514] Processing DeleteTablet for tablet 08130e558b53463daf675faed00da604 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:45:10 UTC) from {username='oryx', principal='oryx/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:45258
I20260806 22:45:10.377678 28898 tablet_service.cc:1514] Processing DeleteTablet for tablet 08130e558b53463daf675faed00da604 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:45:10 UTC) from {username='oryx', principal='oryx/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:47092
I20260806 22:45:10.377962 29290 tablet_replica.cc:330] T 08130e558b53463daf675faed00da604 P 845d3ece65974e25ad6ce260ba27b693: stopping tablet replica
I20260806 22:45:10.378100 29290 raft_consensus.cc:2229] T 08130e558b53463daf675faed00da604 P 845d3ece65974e25ad6ce260ba27b693 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:45:10.378090 29104 tablet_service.cc:1514] Processing DeleteTablet for tablet 08130e558b53463daf675faed00da604 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:45:10 UTC) from {username='oryx', principal='oryx/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:34192
I20260806 22:45:10.378229 29290 raft_consensus.cc:2258] T 08130e558b53463daf675faed00da604 P 845d3ece65974e25ad6ce260ba27b693 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:45:10.378419 29292 tablet_replica.cc:330] T 08130e558b53463daf675faed00da604 P 06ca1eb660804c5f9d85b58de0a16cd2: stopping tablet replica
I20260806 22:45:10.378737 29292 raft_consensus.cc:2229] T 08130e558b53463daf675faed00da604 P 06ca1eb660804c5f9d85b58de0a16cd2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:45:10.378902 29292 raft_consensus.cc:2258] T 08130e558b53463daf675faed00da604 P 06ca1eb660804c5f9d85b58de0a16cd2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:45:10.379123 29290 ts_tablet_manager.cc:1905] T 08130e558b53463daf675faed00da604 P 845d3ece65974e25ad6ce260ba27b693: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:45:10.379238 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 27383
I20260806 22:45:10.387544 29292 ts_tablet_manager.cc:1905] T 08130e558b53463daf675faed00da604 P 06ca1eb660804c5f9d85b58de0a16cd2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:45:10.389860 29292 ts_tablet_manager.cc:1918] T 08130e558b53463daf675faed00da604 P 06ca1eb660804c5f9d85b58de0a16cd2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:45:10.389999 29292 log.cc:1196] T 08130e558b53463daf675faed00da604 P 06ca1eb660804c5f9d85b58de0a16cd2: Deleting WAL directory at /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-2/wal/wals/08130e558b53463daf675faed00da604
I20260806 22:45:10.390319 29292 ts_tablet_manager.cc:1939] T 08130e558b53463daf675faed00da604 P 06ca1eb660804c5f9d85b58de0a16cd2: Deleting consensus metadata
I20260806 22:45:10.391489 29287 tablet_replica.cc:330] T 08130e558b53463daf675faed00da604 P 0bc1abeccf9a48f7b80fcde36b6c6a1e: stopping tablet replica
I20260806 22:45:10.391702 29287 raft_consensus.cc:2229] T 08130e558b53463daf675faed00da604 P 0bc1abeccf9a48f7b80fcde36b6c6a1e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:45:10.391894 29287 raft_consensus.cc:2258] T 08130e558b53463daf675faed00da604 P 0bc1abeccf9a48f7b80fcde36b6c6a1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:45:10.392517 29287 ts_tablet_manager.cc:1905] T 08130e558b53463daf675faed00da604 P 0bc1abeccf9a48f7b80fcde36b6c6a1e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:45:10.393785 29290 ts_tablet_manager.cc:1918] T 08130e558b53463daf675faed00da604 P 845d3ece65974e25ad6ce260ba27b693: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:45:10.394562 29287 ts_tablet_manager.cc:1918] T 08130e558b53463daf675faed00da604 P 0bc1abeccf9a48f7b80fcde36b6c6a1e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:45:10.394654 29287 log.cc:1196] T 08130e558b53463daf675faed00da604 P 0bc1abeccf9a48f7b80fcde36b6c6a1e: Deleting WAL directory at /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-0/wal/wals/08130e558b53463daf675faed00da604
I20260806 22:45:10.394966 29287 ts_tablet_manager.cc:1939] T 08130e558b53463daf675faed00da604 P 0bc1abeccf9a48f7b80fcde36b6c6a1e: Deleting consensus metadata
I20260806 22:45:10.396471 29290 log.cc:1196] T 08130e558b53463daf675faed00da604 P 845d3ece65974e25ad6ce260ba27b693: Deleting WAL directory at /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-1/wal/wals/08130e558b53463daf675faed00da604
I20260806 22:45:10.396939 29290 ts_tablet_manager.cc:1939] T 08130e558b53463daf675faed00da604 P 845d3ece65974e25ad6ce260ba27b693: Deleting consensus metadata
W20260806 22:45:10.400116 28638 connection.cc:535] server connection from 127.0.0.1:45258 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 22:45:10.400125 29032 connection.cc:535] server connection from 127.0.0.1:34192 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 22:45:10.400394 28829 connection.cc:535] server connection from 127.0.0.1:47092 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 22:45:10.402083 29227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:10.377521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52790 (local address 127.18.45.252:40869)
0806 22:45:10.377626 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:10.377628 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:10.391257 (+ 13629us) server_negotiation.cc:407] Connection header received
0806 22:45:10.391356 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:10.391357 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:10.391396 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:10.391459 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:10.394552 (+ 3093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:10.395478 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:10.396386 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:10.396721 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:10.396803 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:10.398570 (+ 1767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:10.398572 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:10.398572 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:10.398584 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:10.400364 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:10.401158 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:10.401160 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:10.401161 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:10.401231 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:10.401451 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:10.401453 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:10.401453 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:10.401601 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:10.401722 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:10.401831 (+ 109us) server_negotiation.cc:299] Negotiation successful
0806 22:45:10.401948 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 22:45:10.411288 28802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:10.402697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52802 (local address 127.18.45.252:40869)
0806 22:45:10.402751 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:10.402754 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:10.403302 (+ 548us) server_negotiation.cc:407] Connection header received
0806 22:45:10.403339 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:10.403340 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:10.403353 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:10.403436 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:10.404389 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:10.405162 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:10.406224 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:10.406523 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:10.406587 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:10.408167 (+ 1580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:10.408168 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:10.408169 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:10.408177 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:10.409940 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:10.410364 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:10.410366 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:10.410366 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:10.410449 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:10.410645 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:10.410647 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:10.410648 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:10.410795 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:10.410924 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:10.411048 (+ 124us) server_negotiation.cc:299] Negotiation successful
0806 22:45:10.411144 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 22:45:10.411876 29226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:10.403108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55818 (local address 127.18.45.253:40509)
0806 22:45:10.403165 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:10.403168 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:10.403179 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:45:10.403215 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:10.403216 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:10.403235 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:10.403302 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:10.404307 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:10.404969 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:10.405969 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:10.406334 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:10.406406 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:10.408220 (+ 1814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:10.408221 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:10.408222 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:10.408232 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:10.410196 (+ 1964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:10.410566 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:10.410567 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:10.410567 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:10.410641 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:10.410858 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:10.410859 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:10.410859 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:10.411057 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:10.411153 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:10.411652 (+ 499us) server_negotiation.cc:299] Negotiation successful
0806 22:45:10.411767 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 22:45:11.371888 29304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:11.371395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39045 (local address 127.18.45.193:36215)
0806 22:45:11.371659 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:11.371789 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":127,"threads_started":1}
W20260806 22:45:11.372254 28797 heartbeater.cc:644] Failed to heartbeat to 127.18.45.254:39045 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.18.45.254:39045: connect: Connection refused (error 111)
I20260806 22:45:11.373078 29305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:11.372759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39045 (local address 127.18.45.194:54115)
0806 22:45:11.372888 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:11.372985 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":72,"threads_started":1}
I20260806 22:45:11.373102 29306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:11.372802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39045 (local address 127.18.45.195:57153)
0806 22:45:11.372947 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:11.373040 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":73,"threads_started":1}
W20260806 22:45:11.373435 29200 heartbeater.cc:644] Failed to heartbeat to 127.18.45.254:39045 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.18.45.254:39045: connect: Connection refused (error 111)
W20260806 22:45:11.373664 28999 heartbeater.cc:644] Failed to heartbeat to 127.18.45.254:39045 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.18.45.254:39045: connect: Connection refused (error 111)
I20260806 22:45:11.876850 29307 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bc26ea2c399a41b3857e7d7f2a8baef6)
I20260806 22:45:11.876919 29308 raft_consensus.cc:482] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bc26ea2c399a41b3857e7d7f2a8baef6)
I20260806 22:45:11.877138 29308 raft_consensus.cc:504] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39045 } } peers { permanent_uuid: "8f1bacf8b0b7450f8cff03a77e4c0890" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 40509 } attrs { promote: false } } peers { permanent_uuid: "69b1737c4ee248689c79e5644bc1274d" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 40869 } attrs { promote: false } }
I20260806 22:45:11.877135 29307 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39045 } } peers { permanent_uuid: "8f1bacf8b0b7450f8cff03a77e4c0890" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 40509 } attrs { promote: false } } peers { permanent_uuid: "69b1737c4ee248689c79e5644bc1274d" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 40869 } attrs { promote: false } }
I20260806 22:45:11.877542 29307 leader_election.cc:288] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bc26ea2c399a41b3857e7d7f2a8baef6 (127.18.45.254:39045), 69b1737c4ee248689c79e5644bc1274d (127.18.45.252:40869)
I20260806 22:45:11.877837 29308 leader_election.cc:288] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bc26ea2c399a41b3857e7d7f2a8baef6 (127.18.45.254:39045), 8f1bacf8b0b7450f8cff03a77e4c0890 (127.18.45.253:40509)
I20260806 22:45:11.878026 29314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:11.877712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39045 (local address 127.0.0.1:56578)
0806 22:45:11.877870 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:11.877965 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":74,"threads_started":1}
W20260806 22:45:11.878188 28254 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39045: connect: Connection refused (error 111)
I20260806 22:45:11.878391 29312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:11.877712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39045 (local address 127.0.0.1:56576)
0806 22:45:11.878272 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:11.878332 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":84,"threads_started":1}
W20260806 22:45:11.878538 28519 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39045: connect: Connection refused (error 111)
I20260806 22:45:11.878682 29317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:11.878565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39045 (local address 127.0.0.1:56606)
0806 22:45:11.878608 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:11.878647 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260806 22:45:11.878772 28254 leader_election.cc:334] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bc26ea2c399a41b3857e7d7f2a8baef6 (127.18.45.254:39045): Network error: Client connection negotiation failed: client connection to 127.18.45.254:39045: connect: Connection refused (error 111)
I20260806 22:45:11.879109 29312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:11.878929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39045 (local address 127.0.0.1:56612)
0806 22:45:11.879032 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:11.879077 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260806 22:45:11.879268 28519 leader_election.cc:334] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bc26ea2c399a41b3857e7d7f2a8baef6 (127.18.45.254:39045): Network error: Client connection negotiation failed: client connection to 127.18.45.254:39045: connect: Connection refused (error 111)
I20260806 22:45:11.882294 29316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:11.877725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.253:40509 (local address 127.0.0.1:55822)
0806 22:45:11.878371 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:11.878382 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:11.878481 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:11.878723 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:11.878724 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:11.878727 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:45:11.878809 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:11.878812 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:11.879758 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:11.879759 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:11.881131 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:11.881135 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:11.882029 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:11.882030 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:11.882096 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:11.882100 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:45:11.882159 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 22:45:11.882214 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":634,"thread_start_us":66,"threads_started":1}
I20260806 22:45:11.882292 29314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:11.877964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.252:40869 (local address 127.0.0.1:52808)
0806 22:45:11.878105 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:11.878118 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:11.878203 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:11.878916 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:11.878918 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:11.878920 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:45:11.878999 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:11.879002 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:11.879804 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:11.879805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:11.881159 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:11.881162 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:11.882016 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:11.882017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:11.882091 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:11.882097 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 22:45:11.882159 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 22:45:11.882210 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"mutex_wait_us":2,"thread_start_us":88,"threads_started":1}
I20260806 22:45:11.882500 29323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:11.878161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52808 (local address 127.18.45.252:40869)
0806 22:45:11.878568 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:11.878571 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:11.878582 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:45:11.878656 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:11.878657 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:11.878669 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:11.879099 (+ 430us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:45:11.879143 (+ 44us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:11.879745 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:11.881239 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:11.881894 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:11.881996 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:11.882010 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:11.882220 (+ 210us) server_negotiation.cc:299] Negotiation successful
0806 22:45:11.882271 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":54,"threads_started":1}
I20260806 22:45:11.882572 29318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:11.878062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55822 (local address 127.18.45.253:40509)
0806 22:45:11.878331 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:11.878333 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:11.878447 (+ 114us) server_negotiation.cc:407] Connection header received
0806 22:45:11.878564 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:11.878565 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:11.878578 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:11.878624 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:45:11.878899 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:11.879671 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:11.881249 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:11.881853 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:11.881985 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:11.882004 (+ 19us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:11.882220 (+ 216us) server_negotiation.cc:299] Negotiation successful
0806 22:45:11.882271 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":253,"thread_start_us":54,"threads_started":1}
I20260806 22:45:11.882753 28555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8f1bacf8b0b7450f8cff03a77e4c0890" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "69b1737c4ee248689c79e5644bc1274d" is_pre_election: true
I20260806 22:45:11.882843 28289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "69b1737c4ee248689c79e5644bc1274d" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "8f1bacf8b0b7450f8cff03a77e4c0890" is_pre_election: true
I20260806 22:45:11.882921 28555 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f1bacf8b0b7450f8cff03a77e4c0890 in term 1.
I20260806 22:45:11.882983 28289 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 69b1737c4ee248689c79e5644bc1274d in term 1.
I20260806 22:45:11.883142 28257 leader_election.cc:302] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 69b1737c4ee248689c79e5644bc1274d, 8f1bacf8b0b7450f8cff03a77e4c0890; no voters: bc26ea2c399a41b3857e7d7f2a8baef6
I20260806 22:45:11.883253 28520 leader_election.cc:302] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 69b1737c4ee248689c79e5644bc1274d, 8f1bacf8b0b7450f8cff03a77e4c0890; no voters: bc26ea2c399a41b3857e7d7f2a8baef6
I20260806 22:45:11.883307 29307 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 22:45:11.883351 29308 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 22:45:11.883369 29307 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890 [term 1 FOLLOWER]: Starting leader election (detected failure of leader bc26ea2c399a41b3857e7d7f2a8baef6)
I20260806 22:45:11.883387 29308 raft_consensus.cc:482] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d [term 1 FOLLOWER]: Starting leader election (detected failure of leader bc26ea2c399a41b3857e7d7f2a8baef6)
I20260806 22:45:11.883390 29307 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890 [term 1 FOLLOWER]: Advancing to term 2
I20260806 22:45:11.883409 29308 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d [term 1 FOLLOWER]: Advancing to term 2
I20260806 22:45:11.884301 29307 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39045 } } peers { permanent_uuid: "8f1bacf8b0b7450f8cff03a77e4c0890" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 40509 } attrs { promote: false } } peers { permanent_uuid: "69b1737c4ee248689c79e5644bc1274d" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 40869 } attrs { promote: false } }
I20260806 22:45:11.884303 29308 raft_consensus.cc:504] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39045 } } peers { permanent_uuid: "8f1bacf8b0b7450f8cff03a77e4c0890" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 40509 } attrs { promote: false } } peers { permanent_uuid: "69b1737c4ee248689c79e5644bc1274d" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 40869 } attrs { promote: false } }
I20260806 22:45:11.884466 29308 leader_election.cc:288] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d [CANDIDATE]: Term 2 election: Requested vote from peers bc26ea2c399a41b3857e7d7f2a8baef6 (127.18.45.254:39045), 8f1bacf8b0b7450f8cff03a77e4c0890 (127.18.45.253:40509)
I20260806 22:45:11.884472 29307 leader_election.cc:288] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890 [CANDIDATE]: Term 2 election: Requested vote from peers bc26ea2c399a41b3857e7d7f2a8baef6 (127.18.45.254:39045), 69b1737c4ee248689c79e5644bc1274d (127.18.45.252:40869)
I20260806 22:45:11.884689 29314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:11.884569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39045 (local address 127.0.0.1:56628)
0806 22:45:11.884622 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:11.884659 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 22:45:11.884718 29316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:11.884569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39045 (local address 127.0.0.1:56626)
0806 22:45:11.884658 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:11.884686 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260806 22:45:11.884732 28555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8f1bacf8b0b7450f8cff03a77e4c0890" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "69b1737c4ee248689c79e5644bc1274d"
I20260806 22:45:11.884779 28289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "69b1737c4ee248689c79e5644bc1274d" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "8f1bacf8b0b7450f8cff03a77e4c0890"
I20260806 22:45:11.884829 28555 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 8f1bacf8b0b7450f8cff03a77e4c0890 in current term 2: Already voted for candidate 69b1737c4ee248689c79e5644bc1274d in this term.
I20260806 22:45:11.884881 28289 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 69b1737c4ee248689c79e5644bc1274d in current term 2: Already voted for candidate 8f1bacf8b0b7450f8cff03a77e4c0890 in this term.
I20260806 22:45:11.885057 29312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:11.884958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39045 (local address 127.0.0.1:56642)
0806 22:45:11.885003 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:11.885029 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260806 22:45:11.885159 28519 leader_election.cc:334] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer bc26ea2c399a41b3857e7d7f2a8baef6 (127.18.45.254:39045): Network error: Client connection negotiation failed: client connection to 127.18.45.254:39045: connect: Connection refused (error 111)
I20260806 22:45:11.885214 28519 leader_election.cc:302] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 69b1737c4ee248689c79e5644bc1274d; no voters: 8f1bacf8b0b7450f8cff03a77e4c0890, bc26ea2c399a41b3857e7d7f2a8baef6
I20260806 22:45:11.885288 29308 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260806 22:45:11.885345 29314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:11.885265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39045 (local address 127.0.0.1:56646)
0806 22:45:11.885295 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:11.885319 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260806 22:45:11.885419 28254 leader_election.cc:334] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer bc26ea2c399a41b3857e7d7f2a8baef6 (127.18.45.254:39045): Network error: Client connection negotiation failed: client connection to 127.18.45.254:39045: connect: Connection refused (error 111)
I20260806 22:45:11.885469 28254 leader_election.cc:302] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8f1bacf8b0b7450f8cff03a77e4c0890; no voters: 69b1737c4ee248689c79e5644bc1274d, bc26ea2c399a41b3857e7d7f2a8baef6
I20260806 22:45:11.885555 29307 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260806 22:45:12.373047 29328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:12.372648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39045 (local address 127.18.45.193:42989)
0806 22:45:12.372849 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:12.372916 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":133,"threads_started":1}
I20260806 22:45:12.374116 29329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:12.373827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39045 (local address 127.18.45.195:37885)
0806 22:45:12.374008 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:12.374049 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":103,"threads_started":1}
I20260806 22:45:12.374277 29330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:12.373964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39045 (local address 127.18.45.194:51375)
0806 22:45:12.374098 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:12.374157 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":76,"threads_started":1}
I20260806 22:45:13.374058 29331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:13.373681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39045 (local address 127.18.45.193:43679)
0806 22:45:13.373903 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:13.373969 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":103,"threads_started":1}
I20260806 22:45:13.374975 29332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:13.374681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39045 (local address 127.18.45.195:35527)
0806 22:45:13.374858 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:13.374909 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":86,"threads_started":1}
I20260806 22:45:13.375190 29333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:13.374681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39045 (local address 127.18.45.194:58733)
0806 22:45:13.375079 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:13.375125 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":88,"threads_started":1}
I20260806 22:45:13.876052 29334 raft_consensus.cc:482] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:45:13.876200 29334 raft_consensus.cc:504] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39045 } } peers { permanent_uuid: "8f1bacf8b0b7450f8cff03a77e4c0890" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 40509 } attrs { promote: false } } peers { permanent_uuid: "69b1737c4ee248689c79e5644bc1274d" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 40869 } attrs { promote: false } }
I20260806 22:45:13.876514 29334 leader_election.cc:288] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers bc26ea2c399a41b3857e7d7f2a8baef6 (127.18.45.254:39045), 8f1bacf8b0b7450f8cff03a77e4c0890 (127.18.45.253:40509)
I20260806 22:45:13.876911 28289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "69b1737c4ee248689c79e5644bc1274d" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "8f1bacf8b0b7450f8cff03a77e4c0890" is_pre_election: true
I20260806 22:45:13.876998 29335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:13.876716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39045 (local address 127.0.0.1:56900)
0806 22:45:13.876892 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:13.876940 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":112,"threads_started":1}
I20260806 22:45:13.877080 28289 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 69b1737c4ee248689c79e5644bc1274d in term 2.
I20260806 22:45:13.877288 28520 leader_election.cc:302] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69b1737c4ee248689c79e5644bc1274d, 8f1bacf8b0b7450f8cff03a77e4c0890; no voters:
I20260806 22:45:13.877432 29334 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260806 22:45:13.877480 29334 raft_consensus.cc:482] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:45:13.877509 29334 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d [term 2 FOLLOWER]: Advancing to term 3
I20260806 22:45:13.877557 29335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:13.877443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39045 (local address 127.0.0.1:56904)
0806 22:45:13.877482 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:13.877515 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
W20260806 22:45:13.877633 28519 leader_election.cc:334] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer bc26ea2c399a41b3857e7d7f2a8baef6 (127.18.45.254:39045): Network error: Client connection negotiation failed: client connection to 127.18.45.254:39045: connect: Connection refused (error 111)
I20260806 22:45:13.878645 29334 raft_consensus.cc:504] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39045 } } peers { permanent_uuid: "8f1bacf8b0b7450f8cff03a77e4c0890" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 40509 } attrs { promote: false } } peers { permanent_uuid: "69b1737c4ee248689c79e5644bc1274d" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 40869 } attrs { promote: false } }
I20260806 22:45:13.878815 29334 leader_election.cc:288] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d [CANDIDATE]: Term 3 election: Requested vote from peers bc26ea2c399a41b3857e7d7f2a8baef6 (127.18.45.254:39045), 8f1bacf8b0b7450f8cff03a77e4c0890 (127.18.45.253:40509)
I20260806 22:45:13.879072 29335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:13.878946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39045 (local address 127.0.0.1:56912)
0806 22:45:13.878992 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:13.879048 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260806 22:45:13.879040 28289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "69b1737c4ee248689c79e5644bc1274d" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "8f1bacf8b0b7450f8cff03a77e4c0890"
I20260806 22:45:13.879158 28289 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890 [term 2 FOLLOWER]: Advancing to term 3
I20260806 22:45:13.879503 29335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:13.879369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39045 (local address 127.0.0.1:56920)
0806 22:45:13.879452 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:13.879478 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260806 22:45:13.879593 28519 leader_election.cc:334] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer bc26ea2c399a41b3857e7d7f2a8baef6 (127.18.45.254:39045): Network error: Client connection negotiation failed: client connection to 127.18.45.254:39045: connect: Connection refused (error 111)
I20260806 22:45:13.880162 28289 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69b1737c4ee248689c79e5644bc1274d in term 3.
I20260806 22:45:13.880376 28520 leader_election.cc:302] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 69b1737c4ee248689c79e5644bc1274d, 8f1bacf8b0b7450f8cff03a77e4c0890; no voters: bc26ea2c399a41b3857e7d7f2a8baef6
I20260806 22:45:13.880489 29334 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d [term 3 FOLLOWER]: Leader election won for term 3
I20260806 22:45:13.880604 29334 raft_consensus.cc:686] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d [term 3 LEADER]: Becoming Leader. State: Replica: 69b1737c4ee248689c79e5644bc1274d, State: Running, Role: LEADER
I20260806 22:45:13.880699 29334 consensus_queue.cc:237] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39045 } } peers { permanent_uuid: "8f1bacf8b0b7450f8cff03a77e4c0890" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 40509 } attrs { promote: false } } peers { permanent_uuid: "69b1737c4ee248689c79e5644bc1274d" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 40869 } attrs { promote: false } }
I20260806 22:45:13.881207 29338 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69b1737c4ee248689c79e5644bc1274d. Latest consensus state: current_term: 3 leader_uuid: "69b1737c4ee248689c79e5644bc1274d" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39045 } } peers { permanent_uuid: "8f1bacf8b0b7450f8cff03a77e4c0890" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 40509 } attrs { promote: false } } peers { permanent_uuid: "69b1737c4ee248689c79e5644bc1274d" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 40869 } attrs { promote: false } } }
I20260806 22:45:13.881333 29338 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d [sys.catalog]: This master's current role is: LEADER
I20260806 22:45:13.881573 29340 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:45:13.882277 29340 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:45:13.882359 29340 catalog_manager.cc:1259] Loaded cluster ID: f71c30cbb9674010be9e9985b316b369
I20260806 22:45:13.882413 29340 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:45:13.882695 29340 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:45:13.882794 29340 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d: Loaded TSK: 0
I20260806 22:45:13.882962 29340 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:45:13.920778 28539 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52802:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:45:13.923072 28539 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:45:13.924614 29335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:13.924442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39045 (local address 127.0.0.1:56934)
0806 22:45:13.924519 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:13.924569 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260806 22:45:13.924752 28289 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890 [term 3 FOLLOWER]: Refusing update from remote peer 69b1737c4ee248689c79e5644bc1274d: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260806 22:45:13.925168 29335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:13.925017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39045 (local address 127.0.0.1:56948)
0806 22:45:13.925093 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:13.925131 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260806 22:45:13.925212 29334 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f1bacf8b0b7450f8cff03a77e4c0890" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 40509 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260806 22:45:13.925493 28519 consensus_peers.cc:487] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d -> Peer bc26ea2c399a41b3857e7d7f2a8baef6 (127.18.45.254:39045): Couldn't send request to peer bc26ea2c399a41b3857e7d7f2a8baef6. Status: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39045: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 22:45:13.926203 29342 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69b1737c4ee248689c79e5644bc1274d. Latest consensus state: current_term: 3 leader_uuid: "69b1737c4ee248689c79e5644bc1274d" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39045 } } peers { permanent_uuid: "8f1bacf8b0b7450f8cff03a77e4c0890" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 40509 } attrs { promote: false } } peers { permanent_uuid: "69b1737c4ee248689c79e5644bc1274d" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 40869 } attrs { promote: false } } }
I20260806 22:45:13.926379 29342 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:45:13.927063 29338 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "69b1737c4ee248689c79e5644bc1274d" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39045 } } peers { permanent_uuid: "8f1bacf8b0b7450f8cff03a77e4c0890" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 40509 } attrs { promote: false } } peers { permanent_uuid: "69b1737c4ee248689c79e5644bc1274d" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 40869 } attrs { promote: false } } }
I20260806 22:45:13.927175 29338 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d [sys.catalog]: This master's current role is: LEADER
I20260806 22:45:13.927270 29342 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "69b1737c4ee248689c79e5644bc1274d" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bc26ea2c399a41b3857e7d7f2a8baef6" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39045 } } peers { permanent_uuid: "8f1bacf8b0b7450f8cff03a77e4c0890" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 40509 } attrs { promote: false } } peers { permanent_uuid: "69b1737c4ee248689c79e5644bc1274d" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 40869 } attrs { promote: false } } }
I20260806 22:45:13.927364 29343 mvcc.cc:205] Tried to move back new op lower bound from 7315686661832282112 to 7315686661656231936. Current Snapshot: MvccSnapshot[applied={T|T < 7315686661832282112}]
I20260806 22:45:13.927944 29342 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f1bacf8b0b7450f8cff03a77e4c0890 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:45:13.933365 29352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:13.929616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47110 (local address 127.18.45.194:43321)
0806 22:45:13.929751 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:13.929753 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:13.929765 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:45:13.929807 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:13.929808 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:13.929822 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:13.930054 (+ 232us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:45:13.930210 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:13.930869 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:13.932252 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:13.932935 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:13.933018 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:13.933036 (+ 18us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:13.933219 (+ 183us) server_negotiation.cc:299] Negotiation successful
0806 22:45:13.933278 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":67,"threads_started":1}
I20260806 22:45:13.933362 29335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:13.929409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.194:43321 (local address 127.0.0.1:47110)
0806 22:45:13.929498 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:13.929512 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:13.929583 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:13.930061 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:13.930062 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:13.930064 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:45:13.930144 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:13.930146 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:13.930964 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:13.930965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:13.932158 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:13.932161 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:13.933037 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:13.933038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:13.933112 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:13.933117 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:45:13.933211 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 22:45:13.933270 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72}
I20260806 22:45:13.933537 29350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:13.929522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.195:44409 (local address 127.0.0.1:34202)
0806 22:45:13.929845 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:13.929856 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:13.929908 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:13.930345 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:13.930346 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:13.930349 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:45:13.930408 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:13.930409 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:13.931369 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:13.931370 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:13.932448 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:13.932450 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:13.933267 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:13.933268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:13.933346 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:13.933351 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:45:13.933399 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 22:45:13.933447 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":82,"threads_started":1}
I20260806 22:45:13.933789 29353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:13.929626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34202 (local address 127.18.45.195:44409)
0806 22:45:13.930182 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:13.930185 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:13.930194 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:45:13.930245 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:13.930246 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:13.930260 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:13.930329 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:45:13.930487 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:13.931284 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:13.932528 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:13.933154 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:13.933218 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:13.933231 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:13.933638 (+ 407us) server_negotiation.cc:299] Negotiation successful
0806 22:45:13.933698 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":523,"thread_start_us":68,"threads_started":1}
I20260806 22:45:13.934154 29355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:13.929686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45262 (local address 127.18.45.193:37419)
0806 22:45:13.929957 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:13.929959 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:13.929968 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:45:13.930000 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:13.930001 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:13.930012 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:13.930085 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:45:13.930268 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:13.931146 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:13.933085 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:13.933742 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:13.933823 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:13.933852 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:13.934047 (+ 195us) server_negotiation.cc:299] Negotiation successful
0806 22:45:13.934090 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":89,"threads_started":1}
I20260806 22:45:13.934172 29349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:13.929409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.193:37419 (local address 127.0.0.1:45262)
0806 22:45:13.929747 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:13.929759 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:13.929819 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:13.930099 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:13.930100 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:13.930103 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:45:13.930163 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:13.930165 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:13.931244 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:13.931245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:13.932165 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:13.932168 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:13.933834 (+ 1666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:13.933835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:13.933963 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:13.933967 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:45:13.934039 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 22:45:13.934094 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"mutex_wait_us":58,"thread_start_us":101,"threads_started":1}
I20260806 22:45:13.934304 28898 tablet_service.cc:1467] Processing CreateTablet for tablet b30c4844be114e5bb6ab99f0550912ba (DEFAULT_TABLE table=test-table [id=772d299f616644639a54f3058094bc6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:45:13.934341 29104 tablet_service.cc:1467] Processing CreateTablet for tablet b30c4844be114e5bb6ab99f0550912ba (DEFAULT_TABLE table=test-table [id=772d299f616644639a54f3058094bc6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:45:13.934507 29104 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b30c4844be114e5bb6ab99f0550912ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:45:13.934507 28898 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b30c4844be114e5bb6ab99f0550912ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:45:13.934484 28699 tablet_service.cc:1467] Processing CreateTablet for tablet b30c4844be114e5bb6ab99f0550912ba (DEFAULT_TABLE table=test-table [id=772d299f616644639a54f3058094bc6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:45:13.934667 28699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b30c4844be114e5bb6ab99f0550912ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:45:13.936340 29358 tablet_bootstrap.cc:492] T b30c4844be114e5bb6ab99f0550912ba P 845d3ece65974e25ad6ce260ba27b693: Bootstrap starting.
I20260806 22:45:13.936396 29359 tablet_bootstrap.cc:492] T b30c4844be114e5bb6ab99f0550912ba P 06ca1eb660804c5f9d85b58de0a16cd2: Bootstrap starting.
I20260806 22:45:13.936713 29357 tablet_bootstrap.cc:492] T b30c4844be114e5bb6ab99f0550912ba P 0bc1abeccf9a48f7b80fcde36b6c6a1e: Bootstrap starting.
I20260806 22:45:13.936959 29359 tablet_bootstrap.cc:654] T b30c4844be114e5bb6ab99f0550912ba P 06ca1eb660804c5f9d85b58de0a16cd2: Neither blocks nor log segments found. Creating new log.
I20260806 22:45:13.936986 29358 tablet_bootstrap.cc:654] T b30c4844be114e5bb6ab99f0550912ba P 845d3ece65974e25ad6ce260ba27b693: Neither blocks nor log segments found. Creating new log.
I20260806 22:45:13.937412 29357 tablet_bootstrap.cc:654] T b30c4844be114e5bb6ab99f0550912ba P 0bc1abeccf9a48f7b80fcde36b6c6a1e: Neither blocks nor log segments found. Creating new log.
I20260806 22:45:13.937690 29359 tablet_bootstrap.cc:492] T b30c4844be114e5bb6ab99f0550912ba P 06ca1eb660804c5f9d85b58de0a16cd2: No bootstrap required, opened a new log
I20260806 22:45:13.937739 29359 ts_tablet_manager.cc:1397] T b30c4844be114e5bb6ab99f0550912ba P 06ca1eb660804c5f9d85b58de0a16cd2: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:45:13.937932 29359 raft_consensus.cc:348] T b30c4844be114e5bb6ab99f0550912ba P 06ca1eb660804c5f9d85b58de0a16cd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bc1abeccf9a48f7b80fcde36b6c6a1e" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 37419 } } peers { permanent_uuid: "845d3ece65974e25ad6ce260ba27b693" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 43321 } } peers { permanent_uuid: "06ca1eb660804c5f9d85b58de0a16cd2" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44409 } }
I20260806 22:45:13.938023 29359 raft_consensus.cc:374] T b30c4844be114e5bb6ab99f0550912ba P 06ca1eb660804c5f9d85b58de0a16cd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:45:13.938047 29359 raft_consensus.cc:729] T b30c4844be114e5bb6ab99f0550912ba P 06ca1eb660804c5f9d85b58de0a16cd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06ca1eb660804c5f9d85b58de0a16cd2, State: Initialized, Role: FOLLOWER
I20260806 22:45:13.938107 29359 consensus_queue.cc:260] T b30c4844be114e5bb6ab99f0550912ba P 06ca1eb660804c5f9d85b58de0a16cd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bc1abeccf9a48f7b80fcde36b6c6a1e" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 37419 } } peers { permanent_uuid: "845d3ece65974e25ad6ce260ba27b693" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 43321 } } peers { permanent_uuid: "06ca1eb660804c5f9d85b58de0a16cd2" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44409 } }
I20260806 22:45:13.938180 29357 tablet_bootstrap.cc:492] T b30c4844be114e5bb6ab99f0550912ba P 0bc1abeccf9a48f7b80fcde36b6c6a1e: No bootstrap required, opened a new log
I20260806 22:45:13.938302 29359 ts_tablet_manager.cc:1428] T b30c4844be114e5bb6ab99f0550912ba P 06ca1eb660804c5f9d85b58de0a16cd2: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 22:45:13.938264 29357 ts_tablet_manager.cc:1397] T b30c4844be114e5bb6ab99f0550912ba P 0bc1abeccf9a48f7b80fcde36b6c6a1e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:45:13.938778 29366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:13.938511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39045 (local address 127.18.45.195:56203)
0806 22:45:13.938690 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:13.938738 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":127,"threads_started":1}
I20260806 22:45:13.939021 29358 tablet_bootstrap.cc:492] T b30c4844be114e5bb6ab99f0550912ba P 845d3ece65974e25ad6ce260ba27b693: No bootstrap required, opened a new log
I20260806 22:45:13.939090 29358 ts_tablet_manager.cc:1397] T b30c4844be114e5bb6ab99f0550912ba P 845d3ece65974e25ad6ce260ba27b693: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 22:45:13.939095 29197 heartbeater.cc:503] Master 127.18.45.252:40869 was elected leader, sending a full tablet report...
I20260806 22:45:13.939254 29358 raft_consensus.cc:348] T b30c4844be114e5bb6ab99f0550912ba P 845d3ece65974e25ad6ce260ba27b693 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bc1abeccf9a48f7b80fcde36b6c6a1e" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 37419 } } peers { permanent_uuid: "845d3ece65974e25ad6ce260ba27b693" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 43321 } } peers { permanent_uuid: "06ca1eb660804c5f9d85b58de0a16cd2" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44409 } }
I20260806 22:45:13.939352 29358 raft_consensus.cc:374] T b30c4844be114e5bb6ab99f0550912ba P 845d3ece65974e25ad6ce260ba27b693 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:45:13.939374 29358 raft_consensus.cc:729] T b30c4844be114e5bb6ab99f0550912ba P 845d3ece65974e25ad6ce260ba27b693 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 845d3ece65974e25ad6ce260ba27b693, State: Initialized, Role: FOLLOWER
I20260806 22:45:13.939378 29366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:13.939154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39045 (local address 127.18.45.195:60471)
0806 22:45:13.939310 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:13.939348 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":107,"threads_started":1}
I20260806 22:45:13.939424 29358 consensus_queue.cc:260] T b30c4844be114e5bb6ab99f0550912ba P 845d3ece65974e25ad6ce260ba27b693 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bc1abeccf9a48f7b80fcde36b6c6a1e" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 37419 } } peers { permanent_uuid: "845d3ece65974e25ad6ce260ba27b693" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 43321 } } peers { permanent_uuid: "06ca1eb660804c5f9d85b58de0a16cd2" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44409 } }
I20260806 22:45:13.939617 29358 ts_tablet_manager.cc:1428] T b30c4844be114e5bb6ab99f0550912ba P 845d3ece65974e25ad6ce260ba27b693: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 22:45:13.939653 29357 raft_consensus.cc:348] T b30c4844be114e5bb6ab99f0550912ba P 0bc1abeccf9a48f7b80fcde36b6c6a1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bc1abeccf9a48f7b80fcde36b6c6a1e" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 37419 } } peers { permanent_uuid: "845d3ece65974e25ad6ce260ba27b693" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 43321 } } peers { permanent_uuid: "06ca1eb660804c5f9d85b58de0a16cd2" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44409 } }
I20260806 22:45:13.939751 29357 raft_consensus.cc:374] T b30c4844be114e5bb6ab99f0550912ba P 0bc1abeccf9a48f7b80fcde36b6c6a1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:45:13.939771 29357 raft_consensus.cc:729] T b30c4844be114e5bb6ab99f0550912ba P 0bc1abeccf9a48f7b80fcde36b6c6a1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bc1abeccf9a48f7b80fcde36b6c6a1e, State: Initialized, Role: FOLLOWER
I20260806 22:45:13.939832 29357 consensus_queue.cc:260] T b30c4844be114e5bb6ab99f0550912ba P 0bc1abeccf9a48f7b80fcde36b6c6a1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bc1abeccf9a48f7b80fcde36b6c6a1e" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 37419 } } peers { permanent_uuid: "845d3ece65974e25ad6ce260ba27b693" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 43321 } } peers { permanent_uuid: "06ca1eb660804c5f9d85b58de0a16cd2" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44409 } }
I20260806 22:45:13.940160 29357 ts_tablet_manager.cc:1428] T b30c4844be114e5bb6ab99f0550912ba P 0bc1abeccf9a48f7b80fcde36b6c6a1e: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:45:13.940254 29374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:13.939983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39045 (local address 127.18.45.194:55593)
0806 22:45:13.940148 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:13.940192 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":104,"threads_started":1}
I20260806 22:45:13.940526 28998 heartbeater.cc:503] Master 127.18.45.252:40869 was elected leader, sending a full tablet report...
I20260806 22:45:13.940681 28795 heartbeater.cc:503] Master 127.18.45.252:40869 was elected leader, sending a full tablet report...
I20260806 22:45:13.940747 29374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:13.940646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39045 (local address 127.18.45.194:48899)
0806 22:45:13.940690 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:13.940718 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260806 22:45:13.941322 29375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:13.940550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39045 (local address 127.18.45.193:55011)
0806 22:45:13.941222 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:13.941267 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":641,"thread_start_us":97,"threads_started":1}
I20260806 22:45:13.941694 29375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:13.941577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39045 (local address 127.18.45.193:45103)
0806 22:45:13.941633 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:13.941660 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 22:45:14.067978 29364 raft_consensus.cc:482] T b30c4844be114e5bb6ab99f0550912ba P 06ca1eb660804c5f9d85b58de0a16cd2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:45:14.068164 29364 raft_consensus.cc:504] T b30c4844be114e5bb6ab99f0550912ba P 06ca1eb660804c5f9d85b58de0a16cd2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bc1abeccf9a48f7b80fcde36b6c6a1e" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 37419 } } peers { permanent_uuid: "845d3ece65974e25ad6ce260ba27b693" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 43321 } } peers { permanent_uuid: "06ca1eb660804c5f9d85b58de0a16cd2" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44409 } }
I20260806 22:45:14.068701 29364 leader_election.cc:288] T b30c4844be114e5bb6ab99f0550912ba P 06ca1eb660804c5f9d85b58de0a16cd2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bc1abeccf9a48f7b80fcde36b6c6a1e (127.18.45.193:37419), 845d3ece65974e25ad6ce260ba27b693 (127.18.45.194:43321)
I20260806 22:45:14.072291 29352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:14.069085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.195:54049 (local address 127.18.45.194:43321)
0806 22:45:14.069154 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:14.069156 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:14.069255 (+ 99us) server_negotiation.cc:407] Connection header received
0806 22:45:14.069361 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:14.069362 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:14.069377 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:14.069456 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:45:14.069693 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:14.070252 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:14.071496 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:14.071894 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:14.071982 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:14.071996 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:14.072145 (+ 149us) server_negotiation.cc:299] Negotiation successful
0806 22:45:14.072194 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 22:45:14.072300 29366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:14.068991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.194:43321 (local address 127.18.45.195:54049)
0806 22:45:14.069144 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:14.069159 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:14.069251 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:14.069461 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:14.069462 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:14.069464 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:45:14.069598 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:14.069600 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:14.070345 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:14.070347 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:14.071389 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:14.071392 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:14.071981 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:14.071982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:14.072061 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:14.072066 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:45:14.072137 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 22:45:14.072192 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"mutex_wait_us":58}
I20260806 22:45:14.072319 29355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:14.069081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.195:42371 (local address 127.18.45.193:37419)
0806 22:45:14.069158 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:14.069161 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:14.069307 (+ 146us) server_negotiation.cc:407] Connection header received
0806 22:45:14.069354 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:14.069355 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:14.069374 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:14.069449 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:45:14.069680 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:14.070253 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:14.071500 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:14.071901 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:14.071995 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:14.072009 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:14.072176 (+ 167us) server_negotiation.cc:299] Negotiation successful
0806 22:45:14.072239 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 22:45:14.072319 29368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:14.068991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.193:37419 (local address 127.18.45.195:42371)
0806 22:45:14.069197 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:14.069207 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:14.069293 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:14.069455 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:14.069456 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:14.069458 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:45:14.069597 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:14.069600 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:14.070360 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:14.070361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:14.071396 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:14.071399 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:14.072027 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:14.072028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:14.072097 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:14.072100 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:45:14.072165 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 22:45:14.072215 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"mutex_wait_us":119}
I20260806 22:45:14.072682 28728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b30c4844be114e5bb6ab99f0550912ba" candidate_uuid: "06ca1eb660804c5f9d85b58de0a16cd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bc1abeccf9a48f7b80fcde36b6c6a1e" is_pre_election: true
I20260806 22:45:14.072696 28923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b30c4844be114e5bb6ab99f0550912ba" candidate_uuid: "06ca1eb660804c5f9d85b58de0a16cd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "845d3ece65974e25ad6ce260ba27b693" is_pre_election: true
I20260806 22:45:14.072834 28728 raft_consensus.cc:2454] T b30c4844be114e5bb6ab99f0550912ba P 0bc1abeccf9a48f7b80fcde36b6c6a1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ca1eb660804c5f9d85b58de0a16cd2 in term 0.
I20260806 22:45:14.072889 28923 raft_consensus.cc:2454] T b30c4844be114e5bb6ab99f0550912ba P 845d3ece65974e25ad6ce260ba27b693 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06ca1eb660804c5f9d85b58de0a16cd2 in term 0.
I20260806 22:45:14.073096 29031 leader_election.cc:302] T b30c4844be114e5bb6ab99f0550912ba P 06ca1eb660804c5f9d85b58de0a16cd2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ca1eb660804c5f9d85b58de0a16cd2, 0bc1abeccf9a48f7b80fcde36b6c6a1e; no voters:
I20260806 22:45:14.073302 29364 raft_consensus.cc:2781] T b30c4844be114e5bb6ab99f0550912ba P 06ca1eb660804c5f9d85b58de0a16cd2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:45:14.073338 29364 raft_consensus.cc:482] T b30c4844be114e5bb6ab99f0550912ba P 06ca1eb660804c5f9d85b58de0a16cd2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:45:14.073352 29364 raft_consensus.cc:3037] T b30c4844be114e5bb6ab99f0550912ba P 06ca1eb660804c5f9d85b58de0a16cd2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:45:14.074265 29364 raft_consensus.cc:504] T b30c4844be114e5bb6ab99f0550912ba P 06ca1eb660804c5f9d85b58de0a16cd2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bc1abeccf9a48f7b80fcde36b6c6a1e" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 37419 } } peers { permanent_uuid: "845d3ece65974e25ad6ce260ba27b693" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 43321 } } peers { permanent_uuid: "06ca1eb660804c5f9d85b58de0a16cd2" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44409 } }
I20260806 22:45:14.074451 29364 leader_election.cc:288] T b30c4844be114e5bb6ab99f0550912ba P 06ca1eb660804c5f9d85b58de0a16cd2 [CANDIDATE]: Term 1 election: Requested vote from peers 0bc1abeccf9a48f7b80fcde36b6c6a1e (127.18.45.193:37419), 845d3ece65974e25ad6ce260ba27b693 (127.18.45.194:43321)
I20260806 22:45:14.074635 28728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b30c4844be114e5bb6ab99f0550912ba" candidate_uuid: "06ca1eb660804c5f9d85b58de0a16cd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bc1abeccf9a48f7b80fcde36b6c6a1e"
I20260806 22:45:14.074746 28728 raft_consensus.cc:3037] T b30c4844be114e5bb6ab99f0550912ba P 0bc1abeccf9a48f7b80fcde36b6c6a1e [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:45:14.074783 28923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b30c4844be114e5bb6ab99f0550912ba" candidate_uuid: "06ca1eb660804c5f9d85b58de0a16cd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "845d3ece65974e25ad6ce260ba27b693"
I20260806 22:45:14.074878 28923 raft_consensus.cc:3037] T b30c4844be114e5bb6ab99f0550912ba P 845d3ece65974e25ad6ce260ba27b693 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:45:14.075771 28728 raft_consensus.cc:2454] T b30c4844be114e5bb6ab99f0550912ba P 0bc1abeccf9a48f7b80fcde36b6c6a1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ca1eb660804c5f9d85b58de0a16cd2 in term 1.
I20260806 22:45:14.076025 29031 leader_election.cc:302] T b30c4844be114e5bb6ab99f0550912ba P 06ca1eb660804c5f9d85b58de0a16cd2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ca1eb660804c5f9d85b58de0a16cd2, 0bc1abeccf9a48f7b80fcde36b6c6a1e; no voters:
I20260806 22:45:14.076093 28923 raft_consensus.cc:2454] T b30c4844be114e5bb6ab99f0550912ba P 845d3ece65974e25ad6ce260ba27b693 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ca1eb660804c5f9d85b58de0a16cd2 in term 1.
I20260806 22:45:14.076197 29364 raft_consensus.cc:2781] T b30c4844be114e5bb6ab99f0550912ba P 06ca1eb660804c5f9d85b58de0a16cd2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:45:14.076444 29364 raft_consensus.cc:686] T b30c4844be114e5bb6ab99f0550912ba P 06ca1eb660804c5f9d85b58de0a16cd2 [term 1 LEADER]: Becoming Leader. State: Replica: 06ca1eb660804c5f9d85b58de0a16cd2, State: Running, Role: LEADER
I20260806 22:45:14.076545 29364 consensus_queue.cc:237] T b30c4844be114e5bb6ab99f0550912ba P 06ca1eb660804c5f9d85b58de0a16cd2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bc1abeccf9a48f7b80fcde36b6c6a1e" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 37419 } } peers { permanent_uuid: "845d3ece65974e25ad6ce260ba27b693" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 43321 } } peers { permanent_uuid: "06ca1eb660804c5f9d85b58de0a16cd2" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44409 } }
I20260806 22:45:14.076894 29366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:14.076721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39045 (local address 127.18.45.195:37399)
0806 22:45:14.076804 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:14.076843 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260806 22:45:14.077575 28539 catalog_manager.cc:5495] T b30c4844be114e5bb6ab99f0550912ba P 06ca1eb660804c5f9d85b58de0a16cd2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 06ca1eb660804c5f9d85b58de0a16cd2 (127.18.45.195). New cstate: current_term: 1 leader_uuid: "06ca1eb660804c5f9d85b58de0a16cd2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bc1abeccf9a48f7b80fcde36b6c6a1e" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 37419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "845d3ece65974e25ad6ce260ba27b693" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 43321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06ca1eb660804c5f9d85b58de0a16cd2" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 44409 } health_report { overall_health: HEALTHY } } }
I20260806 22:45:14.120581 29353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:14.117665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34208 (local address 127.18.45.195:44409)
0806 22:45:14.117737 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:14.117739 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:14.117748 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:45:14.117798 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:14.117799 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:14.117826 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:14.117904 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:45:14.118107 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:14.118647 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:14.119562 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:14.119785 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:14.119870 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:14.120150 (+ 280us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:45:14.120303 (+ 153us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:45:14.120381 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:14.120455 (+ 74us) server_negotiation.cc:299] Negotiation successful
0806 22:45:14.120488 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 22:45:14.122299 28728 raft_consensus.cc:1261] T b30c4844be114e5bb6ab99f0550912ba P 0bc1abeccf9a48f7b80fcde36b6c6a1e [term 1 FOLLOWER]: Refusing update from remote peer 06ca1eb660804c5f9d85b58de0a16cd2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:45:14.122385 28923 raft_consensus.cc:1261] T b30c4844be114e5bb6ab99f0550912ba P 845d3ece65974e25ad6ce260ba27b693 [term 1 FOLLOWER]: Refusing update from remote peer 06ca1eb660804c5f9d85b58de0a16cd2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:45:14.122750 29377 consensus_queue.cc:1035] T b30c4844be114e5bb6ab99f0550912ba P 06ca1eb660804c5f9d85b58de0a16cd2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bc1abeccf9a48f7b80fcde36b6c6a1e" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 37419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:45:14.122920 29364 consensus_queue.cc:1035] T b30c4844be114e5bb6ab99f0550912ba P 06ca1eb660804c5f9d85b58de0a16cd2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "845d3ece65974e25ad6ce260ba27b693" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 43321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:45:14.123991 29375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:14.123727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39045 (local address 127.18.45.193:37107)
0806 22:45:14.123889 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:14.123939 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139}
I20260806 22:45:14.124111 29374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:14.123983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39045 (local address 127.18.45.194:55767)
0806 22:45:14.124036 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:14.124070 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260806 22:45:14.125638 29381 mvcc.cc:205] Tried to move back new op lower bound from 7315686662641500160 to 7315686662458208256. Current Snapshot: MvccSnapshot[applied={T|T < 7315686662641500160}]
I20260806 22:45:14.125864 29383 mvcc.cc:205] Tried to move back new op lower bound from 7315686662641500160 to 7315686662458208256. Current Snapshot: MvccSnapshot[applied={T|T < 7315686662641500160}]
I20260806 22:45:14.126277 29366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:14.126111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39045 (local address 127.18.45.195:38891)
0806 22:45:14.126196 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:14.126233 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260806 22:45:14.126379 29374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:14.126242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39045 (local address 127.18.45.194:56191)
0806 22:45:14.126310 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:14.126340 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 22:45:14.126626 29375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:14.125979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39045 (local address 127.18.45.193:43759)
0806 22:45:14.126555 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:14.126591 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":557}
I20260806 22:45:14.129733 28539 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52802:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:45:14.129874 28539 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52802:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:45:14.131781 28539 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 69b1737c4ee248689c79e5644bc1274d: Sending DeleteTablet for 3 replicas of tablet b30c4844be114e5bb6ab99f0550912ba
I20260806 22:45:14.132278 29104 tablet_service.cc:1514] Processing DeleteTablet for tablet b30c4844be114e5bb6ab99f0550912ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:45:14 UTC) from {username='oryx', principal='oryx/127.18.45.252@KRBTEST.COM'} at 127.0.0.1:34202
I20260806 22:45:14.132437 28699 tablet_service.cc:1514] Processing DeleteTablet for tablet b30c4844be114e5bb6ab99f0550912ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:45:14 UTC) from {username='oryx', principal='oryx/127.18.45.252@KRBTEST.COM'} at 127.0.0.1:45262
I20260806 22:45:14.132647 28898 tablet_service.cc:1514] Processing DeleteTablet for tablet b30c4844be114e5bb6ab99f0550912ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:45:14 UTC) from {username='oryx', principal='oryx/127.18.45.252@KRBTEST.COM'} at 127.0.0.1:47110
I20260806 22:45:14.132824 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 28486
I20260806 22:45:14.133595 29395 tablet_replica.cc:330] T b30c4844be114e5bb6ab99f0550912ba P 845d3ece65974e25ad6ce260ba27b693: stopping tablet replica
I20260806 22:45:14.133715 29395 raft_consensus.cc:2229] T b30c4844be114e5bb6ab99f0550912ba P 845d3ece65974e25ad6ce260ba27b693 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:45:14.133833 29395 raft_consensus.cc:2258] T b30c4844be114e5bb6ab99f0550912ba P 845d3ece65974e25ad6ce260ba27b693 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:45:14.134002 29392 tablet_replica.cc:330] T b30c4844be114e5bb6ab99f0550912ba P 06ca1eb660804c5f9d85b58de0a16cd2: stopping tablet replica
I20260806 22:45:14.134093 29392 raft_consensus.cc:2229] T b30c4844be114e5bb6ab99f0550912ba P 06ca1eb660804c5f9d85b58de0a16cd2 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:45:14.134239 29392 raft_consensus.cc:2258] T b30c4844be114e5bb6ab99f0550912ba P 06ca1eb660804c5f9d85b58de0a16cd2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:45:14.138926 29395 ts_tablet_manager.cc:1905] T b30c4844be114e5bb6ab99f0550912ba P 845d3ece65974e25ad6ce260ba27b693: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:45:14.140286 29392 ts_tablet_manager.cc:1905] T b30c4844be114e5bb6ab99f0550912ba P 06ca1eb660804c5f9d85b58de0a16cd2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:45:14.141649 29395 ts_tablet_manager.cc:1918] T b30c4844be114e5bb6ab99f0550912ba P 845d3ece65974e25ad6ce260ba27b693: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:45:14.141737 29395 log.cc:1196] T b30c4844be114e5bb6ab99f0550912ba P 845d3ece65974e25ad6ce260ba27b693: Deleting WAL directory at /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-1/wal/wals/b30c4844be114e5bb6ab99f0550912ba
I20260806 22:45:14.142052 29395 ts_tablet_manager.cc:1939] T b30c4844be114e5bb6ab99f0550912ba P 845d3ece65974e25ad6ce260ba27b693: Deleting consensus metadata
I20260806 22:45:14.142800 29392 ts_tablet_manager.cc:1918] T b30c4844be114e5bb6ab99f0550912ba P 06ca1eb660804c5f9d85b58de0a16cd2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:45:14.142890 28521 catalog_manager.cc:4841] TS 845d3ece65974e25ad6ce260ba27b693 (127.18.45.194:43321): tablet b30c4844be114e5bb6ab99f0550912ba (table test-table [id=772d299f616644639a54f3058094bc6e]) successfully deleted
I20260806 22:45:14.142940 29392 log.cc:1196] T b30c4844be114e5bb6ab99f0550912ba P 06ca1eb660804c5f9d85b58de0a16cd2: Deleting WAL directory at /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786056275803658-18615-0/minicluster-data/ts-2/wal/wals/b30c4844be114e5bb6ab99f0550912ba
I20260806 22:45:14.143357 29392 ts_tablet_manager.cc:1939] T b30c4844be114e5bb6ab99f0550912ba P 06ca1eb660804c5f9d85b58de0a16cd2: Deleting consensus metadata
I20260806 22:45:14.143846 28521 catalog_manager.cc:4841] TS 06ca1eb660804c5f9d85b58de0a16cd2 (127.18.45.195:44409): tablet b30c4844be114e5bb6ab99f0550912ba (table test-table [id=772d299f616644639a54f3058094bc6e]) successfully deleted
I20260806 22:45:14.153034 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 28806
I20260806 22:45:14.153054 29349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:14.152854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.193:37419 (local address 127.0.0.1:45268)
0806 22:45:14.152934 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:14.152982 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.193:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260806 22:45:14.158442 28519 catalog_manager.cc:4584] TS 0bc1abeccf9a48f7b80fcde36b6c6a1e (127.18.45.193:37419): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b30c4844be114e5bb6ab99f0550912ba: Network error: Client connection negotiation failed: client connection to 127.18.45.193:37419: connect: Connection refused (error 111)
I20260806 22:45:14.172315 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 29007
I20260806 22:45:14.187708 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 28083
I20260806 22:45:14.202124 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 28349
2026-08-06T22:45:14Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (7435 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:45:14 dist-test-slave-h3ft krb5kdc[29399](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:45:14 dist-test-slave-h3ft krb5kdc[29399](info): set up 2 sockets
Aug 06 22:45:14 dist-test-slave-h3ft krb5kdc[29399](info): commencing operation
krb5kdc: starting...
W20260806 22:45:15.516084 18615 mini_kdc.cc:121] Time spent starting KDC: real 1.278s user 0.001s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Aug 06 22:45:15 dist-test-slave-h3ft krb5kdc[29399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056315, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:45:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:45:15Z Disabled control of system clock
WARNING: no policy specified for oryx/127.18.45.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.45.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.45.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/krb5kdc/oryx_127.18.45.254.keytab.
Entry for principal oryx/127.18.45.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/krb5kdc/oryx_127.18.45.254.keytab.
I20260806 22:45:15.671437 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:41637
--webserver_interface=127.18.45.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/krb5kdc/oryx_127.18.45.254.keytab
--principal=oryx/127.18.45.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:40761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.45.254:41637
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:45:15.774111 29419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:45:15.774452 29419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:45:15.774498 29419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:45:15.775959 29419 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:45:15.776039 29419 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 22:45:15.776063 29419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:45:15.776088 29419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:45:15.776116 29419 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:45:15.776144 29419 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:45:15.778132 29419 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:40761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.45.254:41637
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/krb5kdc/oryx_127.18.45.254.keytab
--principal=oryx/127.18.45.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:41637
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.45.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29419
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:45:15.778503 29419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:45:15.778803 29419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:45:15.784943 29424 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:15.784946 29425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:15.784946 29429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:15.785360 29419 server_base.cc:1034] running on GCE node
I20260806 22:45:15.785725 29419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:45:15.786167 29419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:45:15.787348 29419 hybrid_clock.cc:648] HybridClock initialized: now 1786056315787332 us; error 51 us; skew 500 ppm
Aug 06 22:45:15 dist-test-slave-h3ft krb5kdc[29399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056315, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.45.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:45:15.790745 29419 init.cc:377] Logged in from keytab as oryx/127.18.45.254@KRBTEST.COM (short username oryx)
I20260806 22:45:15.791437 29419 webserver.cc:460] Webserver started at http://127.18.45.254:39901/ using document root <none> and password file <none>
I20260806 22:45:15.791728 29419 fs_manager.cc:359] Metadata directory not provided
I20260806 22:45:15.791806 29419 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:45:15.791985 29419 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:45:15.793352 29419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/master-0/data/instance:
uuid: "f3e29cbdebbb43999c5d7cf6c549f7b8"
format_stamp: "Formatted at 2026-08-06 22:45:15 on dist-test-slave-h3ft"
server_key: "8d431f6e9aa7501584ba648df8df7543"
server_key_iv: "dfbb495dc8d1c572bf5a451de7cac9d0"
server_key_version: "encryptionkey@0"
I20260806 22:45:15.793805 29419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/master-0/wal/instance:
uuid: "f3e29cbdebbb43999c5d7cf6c549f7b8"
format_stamp: "Formatted at 2026-08-06 22:45:15 on dist-test-slave-h3ft"
server_key: "8d431f6e9aa7501584ba648df8df7543"
server_key_iv: "dfbb495dc8d1c572bf5a451de7cac9d0"
server_key_version: "encryptionkey@0"
I20260806 22:45:15.795583 29419 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.003s
I20260806 22:45:15.796777 29439 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:15.797297 29419 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:45:15.797461 29419 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/master-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/master-0/wal
uuid: "f3e29cbdebbb43999c5d7cf6c549f7b8"
format_stamp: "Formatted at 2026-08-06 22:45:15 on dist-test-slave-h3ft"
server_key: "8d431f6e9aa7501584ba648df8df7543"
server_key_iv: "dfbb495dc8d1c572bf5a451de7cac9d0"
server_key_version: "encryptionkey@0"
I20260806 22:45:15.797571 29419 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:45:15.822319 29419 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:45:15.826752 29419 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:45:15.827164 29419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:45:15.834009 29419 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.254:41637
I20260806 22:45:15.834043 29530 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.254:41637 every 8 connection(s)
I20260806 22:45:15.834569 29419 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
I20260806 22:45:15.835539 29531 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:45:15.838526 29531 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3e29cbdebbb43999c5d7cf6c549f7b8: Bootstrap starting.
I20260806 22:45:15.839350 29531 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f3e29cbdebbb43999c5d7cf6c549f7b8: Neither blocks nor log segments found. Creating new log.
I20260806 22:45:15.839745 29531 log.cc:824] T 00000000000000000000000000000000 P f3e29cbdebbb43999c5d7cf6c549f7b8: Log is configured to *not* fsync() on all Append() calls
I20260806 22:45:15.839784 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 29419
I20260806 22:45:15.839895 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/master-0/wal/instance
I20260806 22:45:15.840157 18615 external_mini_cluster.cc:1428] Setting key a7693544b08d7a3fae904ea7d2f55f69
I20260806 22:45:15.840708 29531 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3e29cbdebbb43999c5d7cf6c549f7b8: No bootstrap required, opened a new log
I20260806 22:45:15.844493 29531 raft_consensus.cc:348] T 00000000000000000000000000000000 P f3e29cbdebbb43999c5d7cf6c549f7b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3e29cbdebbb43999c5d7cf6c549f7b8" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 41637 } }
I20260806 22:45:15.844724 29531 raft_consensus.cc:374] T 00000000000000000000000000000000 P f3e29cbdebbb43999c5d7cf6c549f7b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:45:15.844775 29531 raft_consensus.cc:729] T 00000000000000000000000000000000 P f3e29cbdebbb43999c5d7cf6c549f7b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3e29cbdebbb43999c5d7cf6c549f7b8, State: Initialized, Role: FOLLOWER
I20260806 22:45:15.845346 29531 consensus_queue.cc:260] T 00000000000000000000000000000000 P f3e29cbdebbb43999c5d7cf6c549f7b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3e29cbdebbb43999c5d7cf6c549f7b8" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 41637 } }
I20260806 22:45:15.845517 29531 raft_consensus.cc:388] T 00000000000000000000000000000000 P f3e29cbdebbb43999c5d7cf6c549f7b8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:45:15.845571 29531 raft_consensus.cc:482] T 00000000000000000000000000000000 P f3e29cbdebbb43999c5d7cf6c549f7b8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:45:15.845640 29531 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f3e29cbdebbb43999c5d7cf6c549f7b8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:45:15.846873 29531 raft_consensus.cc:504] T 00000000000000000000000000000000 P f3e29cbdebbb43999c5d7cf6c549f7b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3e29cbdebbb43999c5d7cf6c549f7b8" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 41637 } }
I20260806 22:45:15.847074 29531 leader_election.cc:302] T 00000000000000000000000000000000 P f3e29cbdebbb43999c5d7cf6c549f7b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f3e29cbdebbb43999c5d7cf6c549f7b8; no voters:
I20260806 22:45:15.847306 29531 leader_election.cc:288] T 00000000000000000000000000000000 P f3e29cbdebbb43999c5d7cf6c549f7b8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:45:15.847465 29537 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f3e29cbdebbb43999c5d7cf6c549f7b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:45:15.847704 29531 sys_catalog.cc:564] T 00000000000000000000000000000000 P f3e29cbdebbb43999c5d7cf6c549f7b8 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:45:15.847759 29537 raft_consensus.cc:686] T 00000000000000000000000000000000 P f3e29cbdebbb43999c5d7cf6c549f7b8 [term 1 LEADER]: Becoming Leader. State: Replica: f3e29cbdebbb43999c5d7cf6c549f7b8, State: Running, Role: LEADER
I20260806 22:45:15.848003 29537 consensus_queue.cc:237] T 00000000000000000000000000000000 P f3e29cbdebbb43999c5d7cf6c549f7b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3e29cbdebbb43999c5d7cf6c549f7b8" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 41637 } }
I20260806 22:45:15.848541 29537 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3e29cbdebbb43999c5d7cf6c549f7b8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f3e29cbdebbb43999c5d7cf6c549f7b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3e29cbdebbb43999c5d7cf6c549f7b8" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 41637 } } }
I20260806 22:45:15.848659 29537 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3e29cbdebbb43999c5d7cf6c549f7b8 [sys.catalog]: This master's current role is: LEADER
I20260806 22:45:15.848858 29537 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3e29cbdebbb43999c5d7cf6c549f7b8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f3e29cbdebbb43999c5d7cf6c549f7b8. Latest consensus state: current_term: 1 leader_uuid: "f3e29cbdebbb43999c5d7cf6c549f7b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3e29cbdebbb43999c5d7cf6c549f7b8" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 41637 } } }
I20260806 22:45:15.848929 29537 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3e29cbdebbb43999c5d7cf6c549f7b8 [sys.catalog]: This master's current role is: LEADER
I20260806 22:45:15.849043 29549 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:45:15.850008 29549 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:45:15.852583 29549 catalog_manager.cc:1347] Generated new cluster ID: f96c1281d4aa48a299eb7bac4cfa8f66
I20260806 22:45:15.852689 29549 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:45:15 dist-test-slave-h3ft krb5kdc[29399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056315, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM
I20260806 22:45:15.859494 29536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:15.840875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53022 (local address 127.18.45.254:41637)
0806 22:45:15.841131 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:15.841143 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:15.841185 (+ 42us) server_negotiation.cc:407] Connection header received
0806 22:45:15.842319 (+ 1134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:15.842332 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:15.842413 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:15.842603 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:15.843750 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:15.845490 (+ 1740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:15.846823 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:15.847595 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:15.847728 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:15.853868 (+ 6140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:15.853875 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:15.853875 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:15.853895 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:15.856598 (+ 2703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:15.857172 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:15.857178 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:15.857178 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:15.857330 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:15.857620 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:15.857621 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:15.857621 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:15.858072 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:15.858287 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:15.858950 (+ 663us) server_negotiation.cc:299] Negotiation successful
0806 22:45:15.859182 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":115,"threads_started":1}
I20260806 22:45:15.881524 29549 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:45:15.882508 29549 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:45:15.898635 29549 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f3e29cbdebbb43999c5d7cf6c549f7b8: Generated new TSK 0
I20260806 22:45:15.899051 29549 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.18.45.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.45.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.45.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/krb5kdc/oryx_127.18.45.193.keytab.
Entry for principal oryx/127.18.45.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/krb5kdc/oryx_127.18.45.193.keytab.
I20260806 22:45:15.940171 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.193:0
--local_ip_for_outbound_sockets=127.18.45.193
--webserver_interface=127.18.45.193
--webserver_port=0
--tserver_master_addrs=127.18.45.254:41637
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/krb5kdc/oryx_127.18.45.193.keytab
--principal=oryx/127.18.45.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:40761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:45:16.041076 29563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:45:16.041374 29563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:45:16.041419 29563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:45:16.042678 29563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:45:16.042749 29563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:45:16.042790 29563 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:45:16.042850 29563 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.193
I20260806 22:45:16.044728 29563 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:40761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/krb5kdc/oryx_127.18.45.193.keytab
--principal=oryx/127.18.45.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.45.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.45.254:41637
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29563
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.193
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:45:16.045068 29563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:45:16.045374 29563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:45:16.051581 29571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:16.051573 29585 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:16.051577 29572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:16.052073 29563 server_base.cc:1034] running on GCE node
I20260806 22:45:16.052420 29563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:45:16.052755 29563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:45:16.053953 29563 hybrid_clock.cc:648] HybridClock initialized: now 1786056316053946 us; error 42 us; skew 500 ppm
Aug 06 22:45:16 dist-test-slave-h3ft krb5kdc[29399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056316, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.45.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:45:16.057294 29563 init.cc:377] Logged in from keytab as oryx/127.18.45.193@KRBTEST.COM (short username oryx)
I20260806 22:45:16.057842 29563 webserver.cc:460] Webserver started at http://127.18.45.193:41255/ using document root <none> and password file <none>
I20260806 22:45:16.058171 29563 fs_manager.cc:359] Metadata directory not provided
I20260806 22:45:16.058250 29563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:45:16.058435 29563 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:45:16.059839 29563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/ts-0/data/instance:
uuid: "3746ae9981e84ed2ae955a063c7f9cde"
format_stamp: "Formatted at 2026-08-06 22:45:16 on dist-test-slave-h3ft"
server_key: "7d7056e6cee58b2467d733b3e1f62f8e"
server_key_iv: "2d5cee1efa7b164f3899f8680220a2dc"
server_key_version: "encryptionkey@0"
I20260806 22:45:16.060279 29563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/ts-0/wal/instance:
uuid: "3746ae9981e84ed2ae955a063c7f9cde"
format_stamp: "Formatted at 2026-08-06 22:45:16 on dist-test-slave-h3ft"
server_key: "7d7056e6cee58b2467d733b3e1f62f8e"
server_key_iv: "2d5cee1efa7b164f3899f8680220a2dc"
server_key_version: "encryptionkey@0"
I20260806 22:45:16.061885 29563 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:45:16.062889 29594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:16.063133 29563 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:45:16.063258 29563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/ts-0/wal
uuid: "3746ae9981e84ed2ae955a063c7f9cde"
format_stamp: "Formatted at 2026-08-06 22:45:16 on dist-test-slave-h3ft"
server_key: "7d7056e6cee58b2467d733b3e1f62f8e"
server_key_iv: "2d5cee1efa7b164f3899f8680220a2dc"
server_key_version: "encryptionkey@0"
I20260806 22:45:16.063344 29563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:45:16.091519 29563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:45:16.094427 29563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:45:16.094667 29563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:45:16.095544 29563 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:45:16.095615 29563 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:16.095655 29563 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:45:16.095678 29563 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 22:45:16 dist-test-slave-h3ft krb5kdc[29399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056316, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.45.193@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM
I20260806 22:45:16.106263 29563 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.193:35205
I20260806 22:45:16.106405 29762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.193:35205 every 8 connection(s)
I20260806 22:45:16.106897 29563 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
I20260806 22:45:16.109397 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 29563
I20260806 22:45:16.109596 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/ts-0/wal/instance
I20260806 22:45:16.109913 18615 external_mini_cluster.cc:1428] Setting key 575a7ccce4cfa10e4dfd1999cbdc05a4
I20260806 22:45:16.110153 29536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:16.096099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:43005 (local address 127.18.45.254:41637)
0806 22:45:16.096192 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:16.096195 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:16.096478 (+ 283us) server_negotiation.cc:407] Connection header received
0806 22:45:16.097138 (+ 660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:16.097139 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:16.097161 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:16.097256 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:16.098361 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:16.099261 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:16.100735 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:16.101174 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:16.101306 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:16.105703 (+ 4397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:16.105706 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:16.105707 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:16.105722 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:16.108086 (+ 2364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:16.108663 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:16.108665 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:16.108665 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:16.108744 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:16.109061 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:16.109062 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:16.109063 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:16.109356 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:16.109454 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:16.109902 (+ 448us) server_negotiation.cc:299] Negotiation successful
0806 22:45:16.110043 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 22:45:16.110487 29610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:16.096073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:41637 (local address 127.18.45.193:43005)
0806 22:45:16.096286 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:16.096353 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:16.097016 (+ 663us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:16.097374 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:16.097381 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:16.097756 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:16.098236 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:16.098248 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:16.099397 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:16.099399 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:16.100579 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:16.100583 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:16.101357 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:16.101358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:16.101561 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:16.102487 (+ 926us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:16.102502 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:16.105527 (+ 3025us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:45:16.108213 (+ 2686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:16.108222 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:16.108225 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:16.108555 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:16.108825 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:16.108826 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:16.108827 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:16.108958 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:16.109450 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:45:16.109457 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:45:16.109740 (+ 283us) client_negotiation.cc:769] Sending connection context
0806 22:45:16.109846 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 22:45:16.110120 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":129,"threads_started":1}
I20260806 22:45:16.111498 29763 heartbeater.cc:347] Connected to a master server at 127.18.45.254:41637
I20260806 22:45:16.111632 29763 heartbeater.cc:464] Registering TS with master...
I20260806 22:45:16.111871 29763 heartbeater.cc:511] Master 127.18.45.254:41637 requested a full tablet report, sending...
I20260806 22:45:16.112615 29466 ts_manager.cc:194] Registered new tserver with Master: 3746ae9981e84ed2ae955a063c7f9cde (127.18.45.193:35205)
I20260806 22:45:16.113682 29466 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.45.193@KRBTEST.COM'} at 127.18.45.193:43005
I20260806 22:45:16.120236 29772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:16.112979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:41637 (local address 127.18.45.193:58877)
0806 22:45:16.113142 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:16.113153 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:16.113216 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:16.113373 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:16.113374 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:16.113637 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:16.113739 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:16.113743 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:16.114843 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:16.114844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:16.115443 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:16.115446 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:16.115861 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:16.115863 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:16.115972 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:16.116618 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:16.116624 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:16.117004 (+ 380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:45:16.118914 (+ 1910us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:16.118915 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:16.118916 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:16.119195 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:16.119436 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:16.119437 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:16.119438 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:16.119515 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:16.119823 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:45:16.119825 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:45:16.119884 (+ 59us) client_negotiation.cc:769] Sending connection context
0806 22:45:16.119964 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 22:45:16.120111 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":85,"threads_started":1}
I20260806 22:45:16.120236 29536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:16.113046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:58877 (local address 127.18.45.254:41637)
0806 22:45:16.113096 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:16.113098 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:16.113219 (+ 121us) server_negotiation.cc:407] Connection header received
0806 22:45:16.113272 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:16.113273 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:16.113286 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:16.113364 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:16.114022 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:16.114736 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:16.115535 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:16.115763 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:16.115859 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:16.117128 (+ 1269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:16.117129 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:16.117130 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:16.117139 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:16.118795 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:16.119282 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:16.119284 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:16.119284 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:16.119360 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:16.119588 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:16.119589 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:16.119590 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:16.119742 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:16.119820 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:16.120005 (+ 185us) server_negotiation.cc:299] Negotiation successful
0806 22:45:16.120111 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
WARNING: no policy specified for oryx/127.18.45.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.45.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.45.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/krb5kdc/oryx_127.18.45.194.keytab.
Entry for principal oryx/127.18.45.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/krb5kdc/oryx_127.18.45.194.keytab.
I20260806 22:45:16.147771 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.194:0
--local_ip_for_outbound_sockets=127.18.45.194
--webserver_interface=127.18.45.194
--webserver_port=0
--tserver_master_addrs=127.18.45.254:41637
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/krb5kdc/oryx_127.18.45.194.keytab
--principal=oryx/127.18.45.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:40761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:45:16.245846 29776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:45:16.246119 29776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:45:16.246150 29776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:45:16.247561 29776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:45:16.247603 29776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:45:16.247624 29776 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:45:16.247642 29776 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.194
I20260806 22:45:16.249444 29776 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:40761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/krb5kdc/oryx_127.18.45.194.keytab
--principal=oryx/127.18.45.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.45.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.45.254:41637
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29776
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.194
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:45:16.249795 29776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:45:16.250037 29776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:45:16.255924 29783 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:16.256002 29786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:16.255918 29781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:16.256299 29776 server_base.cc:1034] running on GCE node
I20260806 22:45:16.256577 29776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:45:16.256882 29776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:45:16.258057 29776 hybrid_clock.cc:648] HybridClock initialized: now 1786056316258040 us; error 40 us; skew 500 ppm
Aug 06 22:45:16 dist-test-slave-h3ft krb5kdc[29399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056316, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.45.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:45:16.261619 29776 init.cc:377] Logged in from keytab as oryx/127.18.45.194@KRBTEST.COM (short username oryx)
I20260806 22:45:16.262250 29776 webserver.cc:460] Webserver started at http://127.18.45.194:46873/ using document root <none> and password file <none>
I20260806 22:45:16.262549 29776 fs_manager.cc:359] Metadata directory not provided
I20260806 22:45:16.262603 29776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:45:16.262768 29776 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:45:16.264238 29776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/ts-1/data/instance:
uuid: "0717372a32fe47d9bc8a698607619b2e"
format_stamp: "Formatted at 2026-08-06 22:45:16 on dist-test-slave-h3ft"
server_key: "b0ee344da12540ddc5f118dd318e91a3"
server_key_iv: "c48e53ed182da80ec5608c6d6a447f9a"
server_key_version: "encryptionkey@0"
I20260806 22:45:16.264711 29776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/ts-1/wal/instance:
uuid: "0717372a32fe47d9bc8a698607619b2e"
format_stamp: "Formatted at 2026-08-06 22:45:16 on dist-test-slave-h3ft"
server_key: "b0ee344da12540ddc5f118dd318e91a3"
server_key_iv: "c48e53ed182da80ec5608c6d6a447f9a"
server_key_version: "encryptionkey@0"
I20260806 22:45:16.266494 29776 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260806 22:45:16.267597 29801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:16.267828 29776 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:45:16.267971 29776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/ts-1/wal
uuid: "0717372a32fe47d9bc8a698607619b2e"
format_stamp: "Formatted at 2026-08-06 22:45:16 on dist-test-slave-h3ft"
server_key: "b0ee344da12540ddc5f118dd318e91a3"
server_key_iv: "c48e53ed182da80ec5608c6d6a447f9a"
server_key_version: "encryptionkey@0"
I20260806 22:45:16.268046 29776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:45:16.286512 29776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:45:16.289609 29776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:45:16.289767 29776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:45:16.290467 29776 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:45:16.290532 29776 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:16.290562 29776 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:45:16.290617 29776 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 22:45:16 dist-test-slave-h3ft krb5kdc[29399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056316, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.45.194@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM
I20260806 22:45:16.300735 29776 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.194:33361
I20260806 22:45:16.301221 29969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.194:33361 every 8 connection(s)
I20260806 22:45:16.301270 29776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
I20260806 22:45:16.303948 29536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:16.291183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:52345 (local address 127.18.45.254:41637)
0806 22:45:16.291276 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:16.291280 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:16.291660 (+ 380us) server_negotiation.cc:407] Connection header received
0806 22:45:16.292702 (+ 1042us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:16.292704 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:16.292724 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:16.292801 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:16.294068 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:16.294994 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:16.296363 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:16.296698 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:16.296775 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:16.300440 (+ 3665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:16.300442 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:16.300442 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:16.300453 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:16.302402 (+ 1949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:16.302852 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:16.302853 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:16.302853 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:16.302941 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:16.303245 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:16.303246 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:16.303246 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:16.303382 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:16.303446 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:16.303759 (+ 313us) server_negotiation.cc:299] Negotiation successful
0806 22:45:16.303835 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 22:45:16.304275 29813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:16.291191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:41637 (local address 127.18.45.194:52345)
0806 22:45:16.291453 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:16.291522 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:16.292529 (+ 1007us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:16.292920 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:16.292926 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:16.293487 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:16.293925 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:16.293937 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:16.295143 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:16.295145 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:16.296241 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:16.296245 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:16.296831 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:16.296832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:16.297054 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:16.297925 (+ 871us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:16.297943 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:16.300284 (+ 2341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:45:16.302484 (+ 2200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:16.302490 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:16.302492 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:16.302774 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:16.302993 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:16.302994 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:16.302994 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:16.303183 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:16.303465 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:45:16.303469 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:45:16.303619 (+ 150us) client_negotiation.cc:769] Sending connection context
0806 22:45:16.303712 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 22:45:16.303925 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":115,"threads_started":1}
I20260806 22:45:16.304915 29970 heartbeater.cc:347] Connected to a master server at 127.18.45.254:41637
I20260806 22:45:16.305052 29970 heartbeater.cc:464] Registering TS with master...
I20260806 22:45:16.305270 29970 heartbeater.cc:511] Master 127.18.45.254:41637 requested a full tablet report, sending...
I20260806 22:45:16.305754 29467 ts_manager.cc:194] Registered new tserver with Master: 0717372a32fe47d9bc8a698607619b2e (127.18.45.194:33361)
I20260806 22:45:16.306202 29467 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.45.194@KRBTEST.COM'} at 127.18.45.194:52345
I20260806 22:45:16.306453 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 29776
I20260806 22:45:16.306622 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/ts-1/wal/instance
I20260806 22:45:16.306879 18615 external_mini_cluster.cc:1428] Setting key 9ac41e678b0f6af7efdb32f71ba4bb89
I20260806 22:45:16.313769 29536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:16.306392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:51363 (local address 127.18.45.254:41637)
0806 22:45:16.306444 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:16.306446 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:16.306605 (+ 159us) server_negotiation.cc:407] Connection header received
0806 22:45:16.306703 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:16.306704 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:16.306716 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:16.306749 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:16.307262 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:16.308025 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:16.308893 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:16.309129 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:16.309214 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:16.310612 (+ 1398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:16.310613 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:16.310614 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:16.310621 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:16.312274 (+ 1653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:16.312762 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:16.312763 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:16.312764 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:16.312868 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:16.313123 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:16.313125 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:16.313125 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:16.313326 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:16.313406 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:16.313611 (+ 205us) server_negotiation.cc:299] Negotiation successful
0806 22:45:16.313685 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 22:45:16.313895 29982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:16.306381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:41637 (local address 127.18.45.194:51363)
0806 22:45:16.306533 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:16.306548 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:16.306633 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:16.306813 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:16.306813 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:16.307089 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:16.307166 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:16.307169 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:16.308142 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:16.308143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:16.308784 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:16.308786 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:16.309227 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:16.309228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:16.309317 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:16.309923 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:16.309931 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:16.310507 (+ 576us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:45:16.312364 (+ 1857us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:16.312366 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:16.312366 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:16.312660 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:16.312937 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:16.312938 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:16.312939 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:16.313025 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:16.313415 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:45:16.313417 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:45:16.313487 (+ 70us) client_negotiation.cc:769] Sending connection context
0806 22:45:16.313588 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 22:45:16.313739 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":92,"threads_started":1}
WARNING: no policy specified for oryx/127.18.45.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.45.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.45.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/krb5kdc/oryx_127.18.45.195.keytab.
Entry for principal oryx/127.18.45.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/krb5kdc/oryx_127.18.45.195.keytab.
I20260806 22:45:16.343111 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.195:0
--local_ip_for_outbound_sockets=127.18.45.195
--webserver_interface=127.18.45.195
--webserver_port=0
--tserver_master_addrs=127.18.45.254:41637
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/krb5kdc/oryx_127.18.45.195.keytab
--principal=oryx/127.18.45.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:40761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:45:16.446329 29986 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:45:16.446625 29986 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:45:16.446658 29986 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:45:16.448365 29986 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:45:16.448428 29986 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:45:16.448457 29986 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 22:45:16.448513 29986 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.195
I20260806 22:45:16.450591 29986 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:40761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/krb5kdc/oryx_127.18.45.195.keytab
--principal=oryx/127.18.45.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.45.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.45.254:41637
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29986
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.195
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:45:16.450932 29986 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:45:16.451268 29986 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:45:16.457684 29993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:16.457692 29992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:16.457701 29995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:16.457947 29986 server_base.cc:1034] running on GCE node
I20260806 22:45:16.458316 29986 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:45:16.458575 29986 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:45:16.459748 29986 hybrid_clock.cc:648] HybridClock initialized: now 1786056316459736 us; error 31 us; skew 500 ppm
Aug 06 22:45:16 dist-test-slave-h3ft krb5kdc[29399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056316, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.45.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:45:16.462972 29986 init.cc:377] Logged in from keytab as oryx/127.18.45.195@KRBTEST.COM (short username oryx)
I20260806 22:45:16.463610 29986 webserver.cc:460] Webserver started at http://127.18.45.195:41489/ using document root <none> and password file <none>
I20260806 22:45:16.463908 29986 fs_manager.cc:359] Metadata directory not provided
I20260806 22:45:16.463979 29986 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:45:16.464144 29986 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:45:16.465413 29986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/ts-2/data/instance:
uuid: "8fe5cd5567524f4d928ba5b93ce819f6"
format_stamp: "Formatted at 2026-08-06 22:45:16 on dist-test-slave-h3ft"
server_key: "71952fbe7fffe5ea9067f1250ce7ed9c"
server_key_iv: "1e4e546608c50e7bb92f83762f9721ac"
server_key_version: "encryptionkey@0"
I20260806 22:45:16.465826 29986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/ts-2/wal/instance:
uuid: "8fe5cd5567524f4d928ba5b93ce819f6"
format_stamp: "Formatted at 2026-08-06 22:45:16 on dist-test-slave-h3ft"
server_key: "71952fbe7fffe5ea9067f1250ce7ed9c"
server_key_iv: "1e4e546608c50e7bb92f83762f9721ac"
server_key_version: "encryptionkey@0"
I20260806 22:45:16.467556 29986 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260806 22:45:16.468405 30007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:16.468590 29986 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:45:16.468703 29986 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/ts-2/wal
uuid: "8fe5cd5567524f4d928ba5b93ce819f6"
format_stamp: "Formatted at 2026-08-06 22:45:16 on dist-test-slave-h3ft"
server_key: "71952fbe7fffe5ea9067f1250ce7ed9c"
server_key_iv: "1e4e546608c50e7bb92f83762f9721ac"
server_key_version: "encryptionkey@0"
I20260806 22:45:16.468784 29986 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:45:16.479645 29986 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:45:16.482548 29986 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:45:16.482796 29986 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:45:16.483701 29986 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:45:16.483767 29986 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:16.483805 29986 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:45:16.483824 29986 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 22:45:16 dist-test-slave-h3ft krb5kdc[29399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056316, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.45.195@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM
I20260806 22:45:16.493439 29986 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.195:37031
I20260806 22:45:16.493883 30167 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.195:37031 every 8 connection(s)
I20260806 22:45:16.493976 29986 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
I20260806 22:45:16.495814 29536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:16.484410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.195:35445 (local address 127.18.45.254:41637)
0806 22:45:16.484502 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:16.484505 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:16.484990 (+ 485us) server_negotiation.cc:407] Connection header received
0806 22:45:16.485969 (+ 979us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:16.485971 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:16.485989 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:16.486028 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:16.487242 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:16.488193 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:16.489338 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:16.489710 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:16.489798 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:16.492585 (+ 2787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:16.492591 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:16.492591 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:16.492601 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:16.494331 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:16.494815 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:16.494817 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:16.494817 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:16.494886 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:16.495179 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:16.495179 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:16.495180 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:16.495310 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:16.495369 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:16.495656 (+ 287us) server_negotiation.cc:299] Negotiation successful
0806 22:45:16.495736 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 22:45:16.496048 30020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:16.484359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:41637 (local address 127.18.45.195:35445)
0806 22:45:16.484792 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:16.484872 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:16.485799 (+ 927us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:16.486272 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:16.486277 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:16.486677 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:16.487091 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:16.487100 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:16.488324 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:16.488324 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:16.489260 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:16.489263 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:16.489846 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:16.489847 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:16.490001 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:16.490677 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:16.490694 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:16.492443 (+ 1749us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:45:16.494454 (+ 2011us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:16.494460 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:16.494463 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:16.494737 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:16.494960 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:16.494961 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:16.494962 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:16.495125 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:16.495379 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:45:16.495385 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:45:16.495523 (+ 138us) client_negotiation.cc:769] Sending connection context
0806 22:45:16.495611 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 22:45:16.495762 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":99,"threads_started":1}
I20260806 22:45:16.496618 30171 heartbeater.cc:347] Connected to a master server at 127.18.45.254:41637
I20260806 22:45:16.496737 30171 heartbeater.cc:464] Registering TS with master...
I20260806 22:45:16.496953 30171 heartbeater.cc:511] Master 127.18.45.254:41637 requested a full tablet report, sending...
I20260806 22:45:16.497432 29466 ts_manager.cc:194] Registered new tserver with Master: 8fe5cd5567524f4d928ba5b93ce819f6 (127.18.45.195:37031)
I20260806 22:45:16.497879 29466 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.45.195@KRBTEST.COM'} at 127.18.45.195:35445
I20260806 22:45:16.501464 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 29986
I20260806 22:45:16.501660 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/ts-2/wal/instance
I20260806 22:45:16.501914 18615 external_mini_cluster.cc:1428] Setting key 5bbf059455d5cfc0ba4ddb0f26cdc7b6
I20260806 22:45:16.503180 18615 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:45:16.504660 29536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:16.498047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.195:33643 (local address 127.18.45.254:41637)
0806 22:45:16.498092 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:16.498094 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:16.498266 (+ 172us) server_negotiation.cc:407] Connection header received
0806 22:45:16.498294 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:16.498294 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:16.498306 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:16.498365 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:16.498723 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:16.499350 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:16.500168 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:16.500362 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:16.500420 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:16.501599 (+ 1179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:16.501601 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:16.501601 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:16.501609 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:16.503387 (+ 1778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:16.503775 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:16.503776 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:16.503777 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:16.503853 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:16.504077 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:16.504078 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:16.504079 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:16.504208 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:16.504295 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:16.504472 (+ 177us) server_negotiation.cc:299] Negotiation successful
0806 22:45:16.504559 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 22:45:16.504678 30180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:16.497985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:41637 (local address 127.18.45.195:33643)
0806 22:45:16.498182 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:16.498192 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:16.498252 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:16.498381 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:16.498381 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:16.498563 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:16.498642 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:16.498644 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:16.499472 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:16.499473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:16.500078 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:16.500080 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:16.500444 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:16.500445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:16.500529 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:16.500979 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:16.500984 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:16.501488 (+ 504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:45:16.503472 (+ 1984us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:16.503473 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:16.503473 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:16.503695 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:16.503934 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:16.503935 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:16.503935 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:16.504007 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:16.504286 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:45:16.504287 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:45:16.504343 (+ 56us) client_negotiation.cc:769] Sending connection context
0806 22:45:16.504424 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 22:45:16.504557 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":77,"threads_started":1}
Aug 06 22:45:16 dist-test-slave-h3ft krb5kdc[29399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056316, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 22:45:16 dist-test-slave-h3ft krb5kdc[29399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:45:16 dist-test-slave-h3ft krb5kdc[29399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
W20260806 22:45:16.531736 29536 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:45:16.525028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53028 (local address 127.18.45.254:41637)
0806 22:45:16.525126 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:16.525128 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:16.525141 (+ 13us) server_negotiation.cc:407] Connection header received
0806 22:45:16.525190 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:16.525191 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:16.525210 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:16.525283 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:16.525956 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:16.526523 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:16.527393 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:16.527605 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:16.527686 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:16.531571 (+ 3885us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53028: BlockingRecv error: recv got EOF from 127.0.0.1:53028 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 06 22:45:16 dist-test-slave-h3ft krb5kdc[29399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056316, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM
I20260806 22:45:16.540643 29536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:16.532761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53042 (local address 127.18.45.254:41637)
0806 22:45:16.532817 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:16.532820 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:16.532830 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:45:16.532911 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:16.532912 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:16.532924 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:16.532993 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:16.533510 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:16.534010 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:16.534742 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:16.534940 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:16.535110 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:16.537573 (+ 2463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:16.537576 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:16.537576 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:16.537584 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:16.539409 (+ 1825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:16.539855 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:16.539856 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:16.539857 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:16.539929 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:16.540173 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:16.540174 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:16.540175 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:16.540305 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:16.540401 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:16.540494 (+ 93us) server_negotiation.cc:299] Negotiation successful
0806 22:45:16.540559 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 22:45:16.543169 29466 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53042:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:45:16.545300 29466 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:45:16.553138 30212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:16.548680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.193:35205 (local address 127.0.0.1:54092)
0806 22:45:16.549215 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:16.549225 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:16.549288 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:16.549530 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:16.549531 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:16.549534 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:45:16.549607 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:16.549609 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:16.550828 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:16.550829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:16.551926 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:16.551929 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:16.552776 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:16.552777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:16.552859 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:16.552879 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 22:45:16.552964 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 22:45:16.553025 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"mutex_wait_us":77,"thread_start_us":82,"threads_started":1}
I20260806 22:45:16.553158 30213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:16.548986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54092 (local address 127.18.45.193:35205)
0806 22:45:16.549225 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:16.549231 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:16.549276 (+ 45us) server_negotiation.cc:407] Connection header received
0806 22:45:16.549418 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:16.549424 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:16.549445 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:16.549532 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:45:16.549690 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:16.550762 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:16.552026 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:16.552686 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:16.552777 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:16.552872 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:16.552979 (+ 107us) server_negotiation.cc:299] Negotiation successful
0806 22:45:16.553020 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":118,"threads_started":1}
I20260806 22:45:16.553489 30206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:16.548440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.195:37031 (local address 127.0.0.1:36598)
0806 22:45:16.548902 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:16.548913 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:16.548977 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:16.549270 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:16.549274 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:16.549277 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:45:16.549366 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:16.549369 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:16.550761 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:16.550762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:16.551954 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:16.551958 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:16.553106 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:16.553107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:16.553209 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:16.553214 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:45:16.553260 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 22:45:16.553407 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"mutex_wait_us":61,"thread_start_us":135,"threads_started":1}
I20260806 22:45:16.554013 30210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:16.548598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36598 (local address 127.18.45.195:37031)
0806 22:45:16.549011 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:16.549017 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:16.549030 (+ 13us) server_negotiation.cc:407] Connection header received
0806 22:45:16.549084 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:16.549087 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:16.549109 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:16.549252 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:45:16.549456 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:16.550659 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:16.552075 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:16.552993 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:16.553701 (+ 708us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:16.553812 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:16.553841 (+ 29us) server_negotiation.cc:299] Negotiation successful
0806 22:45:16.553896 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":120,"threads_started":1}
I20260806 22:45:16.554419 29673 tablet_service.cc:1467] Processing CreateTablet for tablet 55d158eb3be242ad9e8b54580c108c6c (DEFAULT_TABLE table=test-table [id=12a37740cef5430e830a20c9e5f77b1f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:45:16.554822 30207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:16.548408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.194:33361 (local address 127.0.0.1:46334)
0806 22:45:16.548665 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:16.548722 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:16.548805 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:16.549788 (+ 983us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:16.549788 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:16.549790 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:45:16.549828 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:16.549829 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:16.550950 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:16.550951 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:16.553229 (+ 2278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:16.553231 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:16.554557 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:16.554558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:16.554629 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:16.554636 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 22:45:16.554707 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 22:45:16.554750 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"mutex_wait_us":992,"thread_start_us":89,"threads_started":1}
I20260806 22:45:16.554735 30079 tablet_service.cc:1467] Processing CreateTablet for tablet 55d158eb3be242ad9e8b54580c108c6c (DEFAULT_TABLE table=test-table [id=12a37740cef5430e830a20c9e5f77b1f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:45:16.554917 30208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:16.548585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46334 (local address 127.18.45.194:33361)
0806 22:45:16.549522 (+ 937us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:16.549527 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:16.549538 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:45:16.549587 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:16.549590 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:16.549609 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:16.549700 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:45:16.549971 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:16.550840 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:16.553343 (+ 2503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:16.554450 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:16.554557 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:16.554645 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:16.554726 (+ 81us) server_negotiation.cc:299] Negotiation successful
0806 22:45:16.554774 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":875,"thread_start_us":115,"threads_started":1}
I20260806 22:45:16.554921 29673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55d158eb3be242ad9e8b54580c108c6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:45:16.555135 30079 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55d158eb3be242ad9e8b54580c108c6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:45:16.557736 29876 tablet_service.cc:1467] Processing CreateTablet for tablet 55d158eb3be242ad9e8b54580c108c6c (DEFAULT_TABLE table=test-table [id=12a37740cef5430e830a20c9e5f77b1f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:45:16.558144 29876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55d158eb3be242ad9e8b54580c108c6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:45:16.559592 30214 tablet_bootstrap.cc:492] T 55d158eb3be242ad9e8b54580c108c6c P 8fe5cd5567524f4d928ba5b93ce819f6: Bootstrap starting.
I20260806 22:45:16.560086 30215 tablet_bootstrap.cc:492] T 55d158eb3be242ad9e8b54580c108c6c P 3746ae9981e84ed2ae955a063c7f9cde: Bootstrap starting.
I20260806 22:45:16.560549 30214 tablet_bootstrap.cc:654] T 55d158eb3be242ad9e8b54580c108c6c P 8fe5cd5567524f4d928ba5b93ce819f6: Neither blocks nor log segments found. Creating new log.
I20260806 22:45:16.560796 30215 tablet_bootstrap.cc:654] T 55d158eb3be242ad9e8b54580c108c6c P 3746ae9981e84ed2ae955a063c7f9cde: Neither blocks nor log segments found. Creating new log.
I20260806 22:45:16.560986 30214 log.cc:824] T 55d158eb3be242ad9e8b54580c108c6c P 8fe5cd5567524f4d928ba5b93ce819f6: Log is configured to *not* fsync() on all Append() calls
I20260806 22:45:16.561194 30215 log.cc:824] T 55d158eb3be242ad9e8b54580c108c6c P 3746ae9981e84ed2ae955a063c7f9cde: Log is configured to *not* fsync() on all Append() calls
I20260806 22:45:16.562027 30214 tablet_bootstrap.cc:492] T 55d158eb3be242ad9e8b54580c108c6c P 8fe5cd5567524f4d928ba5b93ce819f6: No bootstrap required, opened a new log
I20260806 22:45:16.562022 30215 tablet_bootstrap.cc:492] T 55d158eb3be242ad9e8b54580c108c6c P 3746ae9981e84ed2ae955a063c7f9cde: No bootstrap required, opened a new log
I20260806 22:45:16.562103 30215 ts_tablet_manager.cc:1397] T 55d158eb3be242ad9e8b54580c108c6c P 3746ae9981e84ed2ae955a063c7f9cde: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:45:16.562103 30214 ts_tablet_manager.cc:1397] T 55d158eb3be242ad9e8b54580c108c6c P 8fe5cd5567524f4d928ba5b93ce819f6: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:45:16.562487 30218 tablet_bootstrap.cc:492] T 55d158eb3be242ad9e8b54580c108c6c P 0717372a32fe47d9bc8a698607619b2e: Bootstrap starting.
I20260806 22:45:16.563550 30218 tablet_bootstrap.cc:654] T 55d158eb3be242ad9e8b54580c108c6c P 0717372a32fe47d9bc8a698607619b2e: Neither blocks nor log segments found. Creating new log.
I20260806 22:45:16.563889 30218 log.cc:824] T 55d158eb3be242ad9e8b54580c108c6c P 0717372a32fe47d9bc8a698607619b2e: Log is configured to *not* fsync() on all Append() calls
I20260806 22:45:16.564435 30214 raft_consensus.cc:348] T 55d158eb3be242ad9e8b54580c108c6c P 8fe5cd5567524f4d928ba5b93ce819f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 37031 } } peers { permanent_uuid: "0717372a32fe47d9bc8a698607619b2e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33361 } } peers { permanent_uuid: "3746ae9981e84ed2ae955a063c7f9cde" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 35205 } }
I20260806 22:45:16.564604 30218 tablet_bootstrap.cc:492] T 55d158eb3be242ad9e8b54580c108c6c P 0717372a32fe47d9bc8a698607619b2e: No bootstrap required, opened a new log
I20260806 22:45:16.564636 30214 raft_consensus.cc:374] T 55d158eb3be242ad9e8b54580c108c6c P 8fe5cd5567524f4d928ba5b93ce819f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:45:16.564677 30214 raft_consensus.cc:729] T 55d158eb3be242ad9e8b54580c108c6c P 8fe5cd5567524f4d928ba5b93ce819f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fe5cd5567524f4d928ba5b93ce819f6, State: Initialized, Role: FOLLOWER
I20260806 22:45:16.564682 30218 ts_tablet_manager.cc:1397] T 55d158eb3be242ad9e8b54580c108c6c P 0717372a32fe47d9bc8a698607619b2e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:45:16.564595 30215 raft_consensus.cc:348] T 55d158eb3be242ad9e8b54580c108c6c P 3746ae9981e84ed2ae955a063c7f9cde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 37031 } } peers { permanent_uuid: "0717372a32fe47d9bc8a698607619b2e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33361 } } peers { permanent_uuid: "3746ae9981e84ed2ae955a063c7f9cde" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 35205 } }
I20260806 22:45:16.564739 30215 raft_consensus.cc:374] T 55d158eb3be242ad9e8b54580c108c6c P 3746ae9981e84ed2ae955a063c7f9cde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:45:16.564771 30215 raft_consensus.cc:729] T 55d158eb3be242ad9e8b54580c108c6c P 3746ae9981e84ed2ae955a063c7f9cde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3746ae9981e84ed2ae955a063c7f9cde, State: Initialized, Role: FOLLOWER
I20260806 22:45:16.564810 30214 consensus_queue.cc:260] T 55d158eb3be242ad9e8b54580c108c6c P 8fe5cd5567524f4d928ba5b93ce819f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 37031 } } peers { permanent_uuid: "0717372a32fe47d9bc8a698607619b2e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33361 } } peers { permanent_uuid: "3746ae9981e84ed2ae955a063c7f9cde" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 35205 } }
I20260806 22:45:16.564903 30215 consensus_queue.cc:260] T 55d158eb3be242ad9e8b54580c108c6c P 3746ae9981e84ed2ae955a063c7f9cde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 37031 } } peers { permanent_uuid: "0717372a32fe47d9bc8a698607619b2e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33361 } } peers { permanent_uuid: "3746ae9981e84ed2ae955a063c7f9cde" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 35205 } }
I20260806 22:45:16.565150 30215 ts_tablet_manager.cc:1428] T 55d158eb3be242ad9e8b54580c108c6c P 3746ae9981e84ed2ae955a063c7f9cde: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:45:16.565141 30214 ts_tablet_manager.cc:1428] T 55d158eb3be242ad9e8b54580c108c6c P 8fe5cd5567524f4d928ba5b93ce819f6: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:45:16.565203 29763 heartbeater.cc:503] Master 127.18.45.254:41637 was elected leader, sending a full tablet report...
I20260806 22:45:16.565289 30171 heartbeater.cc:503] Master 127.18.45.254:41637 was elected leader, sending a full tablet report...
I20260806 22:45:16.567134 30218 raft_consensus.cc:348] T 55d158eb3be242ad9e8b54580c108c6c P 0717372a32fe47d9bc8a698607619b2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 37031 } } peers { permanent_uuid: "0717372a32fe47d9bc8a698607619b2e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33361 } } peers { permanent_uuid: "3746ae9981e84ed2ae955a063c7f9cde" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 35205 } }
I20260806 22:45:16.567303 30218 raft_consensus.cc:374] T 55d158eb3be242ad9e8b54580c108c6c P 0717372a32fe47d9bc8a698607619b2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:45:16.567354 30218 raft_consensus.cc:729] T 55d158eb3be242ad9e8b54580c108c6c P 0717372a32fe47d9bc8a698607619b2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0717372a32fe47d9bc8a698607619b2e, State: Initialized, Role: FOLLOWER
I20260806 22:45:16.567519 30218 consensus_queue.cc:260] T 55d158eb3be242ad9e8b54580c108c6c P 0717372a32fe47d9bc8a698607619b2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 37031 } } peers { permanent_uuid: "0717372a32fe47d9bc8a698607619b2e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33361 } } peers { permanent_uuid: "3746ae9981e84ed2ae955a063c7f9cde" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 35205 } }
I20260806 22:45:16.567860 30218 ts_tablet_manager.cc:1428] T 55d158eb3be242ad9e8b54580c108c6c P 0717372a32fe47d9bc8a698607619b2e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:45:16.567970 29970 heartbeater.cc:503] Master 127.18.45.254:41637 was elected leader, sending a full tablet report...
W20260806 22:45:16.609010 29764 tablet.cc:2369] T 55d158eb3be242ad9e8b54580c108c6c P 3746ae9981e84ed2ae955a063c7f9cde: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:45:16.744884 30172 tablet.cc:2369] T 55d158eb3be242ad9e8b54580c108c6c P 8fe5cd5567524f4d928ba5b93ce819f6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:45:16.767586 30223 raft_consensus.cc:482] T 55d158eb3be242ad9e8b54580c108c6c P 3746ae9981e84ed2ae955a063c7f9cde [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:45:16.767786 30223 raft_consensus.cc:504] T 55d158eb3be242ad9e8b54580c108c6c P 3746ae9981e84ed2ae955a063c7f9cde [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 37031 } } peers { permanent_uuid: "0717372a32fe47d9bc8a698607619b2e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33361 } } peers { permanent_uuid: "3746ae9981e84ed2ae955a063c7f9cde" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 35205 } }
I20260806 22:45:16.768333 30223 leader_election.cc:288] T 55d158eb3be242ad9e8b54580c108c6c P 3746ae9981e84ed2ae955a063c7f9cde [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fe5cd5567524f4d928ba5b93ce819f6 (127.18.45.195:37031), 0717372a32fe47d9bc8a698607619b2e (127.18.45.194:33361)
I20260806 22:45:16.771883 30227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:16.768593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.194:33361 (local address 127.18.45.193:58379)
0806 22:45:16.768809 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:16.768822 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:16.768909 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:16.769073 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:16.769074 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:16.769077 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:45:16.769158 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:16.769161 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:16.769885 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:16.769886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:16.770743 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:16.770746 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:16.771557 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:16.771559 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:16.771632 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:16.771639 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 22:45:16.771698 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 22:45:16.771770 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"mutex_wait_us":59,"thread_start_us":76,"threads_started":1}
I20260806 22:45:16.771888 30208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:16.768753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:58379 (local address 127.18.45.194:33361)
0806 22:45:16.768818 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:16.768821 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:16.768911 (+ 90us) server_negotiation.cc:407] Connection header received
0806 22:45:16.768980 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:16.768981 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:16.768994 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:16.769047 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:45:16.769241 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:16.769799 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:16.770848 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:16.771479 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:16.771543 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:16.771554 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:16.771713 (+ 159us) server_negotiation.cc:299] Negotiation successful
0806 22:45:16.771779 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 22:45:16.772053 30210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:16.768759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:59357 (local address 127.18.45.195:37031)
0806 22:45:16.768823 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:16.768825 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:16.768883 (+ 58us) server_negotiation.cc:407] Connection header received
0806 22:45:16.768993 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:16.768994 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:16.769004 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:16.769089 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:45:16.769307 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:16.769807 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:16.770962 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:16.771603 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:16.771684 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:16.771695 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:16.771889 (+ 194us) server_negotiation.cc:299] Negotiation successful
0806 22:45:16.771959 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 22:45:16.772006 30226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:16.768593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.195:37031 (local address 127.18.45.193:59357)
0806 22:45:16.768807 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:16.768820 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:16.768905 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:16.769111 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:16.769112 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:16.769115 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:45:16.769223 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:16.769226 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:16.769960 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:16.769961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:16.770858 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:16.770860 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:16.771712 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:16.771714 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:16.771803 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:16.771808 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:45:16.771868 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 22:45:16.771924 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"mutex_wait_us":2,"thread_start_us":80,"threads_started":1}
I20260806 22:45:16.772513 30104 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55d158eb3be242ad9e8b54580c108c6c" candidate_uuid: "3746ae9981e84ed2ae955a063c7f9cde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" is_pre_election: true
I20260806 22:45:16.772575 29903 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55d158eb3be242ad9e8b54580c108c6c" candidate_uuid: "3746ae9981e84ed2ae955a063c7f9cde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0717372a32fe47d9bc8a698607619b2e" is_pre_election: true
I20260806 22:45:16.772753 29903 raft_consensus.cc:2454] T 55d158eb3be242ad9e8b54580c108c6c P 0717372a32fe47d9bc8a698607619b2e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3746ae9981e84ed2ae955a063c7f9cde in term 0.
I20260806 22:45:16.772756 30104 raft_consensus.cc:2454] T 55d158eb3be242ad9e8b54580c108c6c P 8fe5cd5567524f4d928ba5b93ce819f6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3746ae9981e84ed2ae955a063c7f9cde in term 0.
I20260806 22:45:16.773048 29602 leader_election.cc:302] T 55d158eb3be242ad9e8b54580c108c6c P 3746ae9981e84ed2ae955a063c7f9cde [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3746ae9981e84ed2ae955a063c7f9cde, 8fe5cd5567524f4d928ba5b93ce819f6; no voters:
I20260806 22:45:16.773231 30223 raft_consensus.cc:2781] T 55d158eb3be242ad9e8b54580c108c6c P 3746ae9981e84ed2ae955a063c7f9cde [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:45:16.773303 30223 raft_consensus.cc:482] T 55d158eb3be242ad9e8b54580c108c6c P 3746ae9981e84ed2ae955a063c7f9cde [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:45:16.773339 30223 raft_consensus.cc:3037] T 55d158eb3be242ad9e8b54580c108c6c P 3746ae9981e84ed2ae955a063c7f9cde [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:45:16.774267 30223 raft_consensus.cc:504] T 55d158eb3be242ad9e8b54580c108c6c P 3746ae9981e84ed2ae955a063c7f9cde [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 37031 } } peers { permanent_uuid: "0717372a32fe47d9bc8a698607619b2e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33361 } } peers { permanent_uuid: "3746ae9981e84ed2ae955a063c7f9cde" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 35205 } }
I20260806 22:45:16.774456 30223 leader_election.cc:288] T 55d158eb3be242ad9e8b54580c108c6c P 3746ae9981e84ed2ae955a063c7f9cde [CANDIDATE]: Term 1 election: Requested vote from peers 8fe5cd5567524f4d928ba5b93ce819f6 (127.18.45.195:37031), 0717372a32fe47d9bc8a698607619b2e (127.18.45.194:33361)
I20260806 22:45:16.774647 29903 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55d158eb3be242ad9e8b54580c108c6c" candidate_uuid: "3746ae9981e84ed2ae955a063c7f9cde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0717372a32fe47d9bc8a698607619b2e"
I20260806 22:45:16.774678 30104 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55d158eb3be242ad9e8b54580c108c6c" candidate_uuid: "3746ae9981e84ed2ae955a063c7f9cde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe5cd5567524f4d928ba5b93ce819f6"
I20260806 22:45:16.774767 30104 raft_consensus.cc:3037] T 55d158eb3be242ad9e8b54580c108c6c P 8fe5cd5567524f4d928ba5b93ce819f6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:45:16.774768 29903 raft_consensus.cc:3037] T 55d158eb3be242ad9e8b54580c108c6c P 0717372a32fe47d9bc8a698607619b2e [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:45:16.775960 29903 raft_consensus.cc:2454] T 55d158eb3be242ad9e8b54580c108c6c P 0717372a32fe47d9bc8a698607619b2e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3746ae9981e84ed2ae955a063c7f9cde in term 1.
I20260806 22:45:16.775964 30104 raft_consensus.cc:2454] T 55d158eb3be242ad9e8b54580c108c6c P 8fe5cd5567524f4d928ba5b93ce819f6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3746ae9981e84ed2ae955a063c7f9cde in term 1.
I20260806 22:45:16.776155 29602 leader_election.cc:302] T 55d158eb3be242ad9e8b54580c108c6c P 3746ae9981e84ed2ae955a063c7f9cde [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3746ae9981e84ed2ae955a063c7f9cde, 8fe5cd5567524f4d928ba5b93ce819f6; no voters:
I20260806 22:45:16.776262 30223 raft_consensus.cc:2781] T 55d158eb3be242ad9e8b54580c108c6c P 3746ae9981e84ed2ae955a063c7f9cde [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:45:16.776391 30223 raft_consensus.cc:686] T 55d158eb3be242ad9e8b54580c108c6c P 3746ae9981e84ed2ae955a063c7f9cde [term 1 LEADER]: Becoming Leader. State: Replica: 3746ae9981e84ed2ae955a063c7f9cde, State: Running, Role: LEADER
I20260806 22:45:16.776518 30223 consensus_queue.cc:237] T 55d158eb3be242ad9e8b54580c108c6c P 3746ae9981e84ed2ae955a063c7f9cde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 37031 } } peers { permanent_uuid: "0717372a32fe47d9bc8a698607619b2e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33361 } } peers { permanent_uuid: "3746ae9981e84ed2ae955a063c7f9cde" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 35205 } }
I20260806 22:45:16.777619 29466 catalog_manager.cc:5495] T 55d158eb3be242ad9e8b54580c108c6c P 3746ae9981e84ed2ae955a063c7f9cde reported cstate change: term changed from 0 to 1, leader changed from <none> to 3746ae9981e84ed2ae955a063c7f9cde (127.18.45.193). New cstate: current_term: 1 leader_uuid: "3746ae9981e84ed2ae955a063c7f9cde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 37031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0717372a32fe47d9bc8a698607619b2e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33361 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3746ae9981e84ed2ae955a063c7f9cde" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 35205 } health_report { overall_health: HEALTHY } } }
W20260806 22:45:16.802829 29974 tablet.cc:2369] T 55d158eb3be242ad9e8b54580c108c6c P 0717372a32fe47d9bc8a698607619b2e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:45:16.840335 30213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:16.837510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54108 (local address 127.18.45.193:35205)
0806 22:45:16.837598 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:16.837600 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:16.837612 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:45:16.837673 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:16.837674 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:16.837705 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:16.837791 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:45:16.838025 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:16.838567 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:16.839420 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:16.839654 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:16.839732 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:16.839942 (+ 210us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:45:16.840065 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:45:16.840155 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:16.840199 (+ 44us) server_negotiation.cc:299] Negotiation successful
0806 22:45:16.840224 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 22:45:16.842919 30104 raft_consensus.cc:1261] T 55d158eb3be242ad9e8b54580c108c6c P 8fe5cd5567524f4d928ba5b93ce819f6 [term 1 FOLLOWER]: Refusing update from remote peer 3746ae9981e84ed2ae955a063c7f9cde: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:45:16.842905 29903 raft_consensus.cc:1261] T 55d158eb3be242ad9e8b54580c108c6c P 0717372a32fe47d9bc8a698607619b2e [term 1 FOLLOWER]: Refusing update from remote peer 3746ae9981e84ed2ae955a063c7f9cde: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:45:16.843431 30223 consensus_queue.cc:1035] T 55d158eb3be242ad9e8b54580c108c6c P 3746ae9981e84ed2ae955a063c7f9cde [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 37031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:45:16.843644 30228 consensus_queue.cc:1035] T 55d158eb3be242ad9e8b54580c108c6c P 3746ae9981e84ed2ae955a063c7f9cde [LEADER]: Connected to new peer: Peer: permanent_uuid: "0717372a32fe47d9bc8a698607619b2e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:45:16.848116 30240 mvcc.cc:205] Tried to move back new op lower bound from 7315686673784188928 to 7315686673517371392. Current Snapshot: MvccSnapshot[applied={T|T < 7315686673784188928}]
I20260806 22:45:16.848992 30233 mvcc.cc:205] Tried to move back new op lower bound from 7315686673784188928 to 7315686673517371392. Current Snapshot: MvccSnapshot[applied={T|T < 7315686673784188928 or (T in {7315686673784188928})}]
I20260806 22:45:16.850239 29467 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53042:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:45:16.850366 29467 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53042:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:45:16.851521 29467 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f3e29cbdebbb43999c5d7cf6c549f7b8: Sending DeleteTablet for 3 replicas of tablet 55d158eb3be242ad9e8b54580c108c6c
I20260806 22:45:16.852075 30079 tablet_service.cc:1514] Processing DeleteTablet for tablet 55d158eb3be242ad9e8b54580c108c6c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:45:16 UTC) from {username='oryx', principal='oryx/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:36598
I20260806 22:45:16.852295 29876 tablet_service.cc:1514] Processing DeleteTablet for tablet 55d158eb3be242ad9e8b54580c108c6c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:45:16 UTC) from {username='oryx', principal='oryx/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:46334
I20260806 22:45:16.852488 29673 tablet_service.cc:1514] Processing DeleteTablet for tablet 55d158eb3be242ad9e8b54580c108c6c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:45:16 UTC) from {username='oryx', principal='oryx/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:54092
I20260806 22:45:16.852864 30249 tablet_replica.cc:330] T 55d158eb3be242ad9e8b54580c108c6c P 0717372a32fe47d9bc8a698607619b2e: stopping tablet replica
I20260806 22:45:16.852982 30249 raft_consensus.cc:2229] T 55d158eb3be242ad9e8b54580c108c6c P 0717372a32fe47d9bc8a698607619b2e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:45:16.853106 30249 raft_consensus.cc:2258] T 55d158eb3be242ad9e8b54580c108c6c P 0717372a32fe47d9bc8a698607619b2e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:45:16.853314 30250 tablet_replica.cc:330] T 55d158eb3be242ad9e8b54580c108c6c P 3746ae9981e84ed2ae955a063c7f9cde: stopping tablet replica
I20260806 22:45:16.853425 30250 raft_consensus.cc:2229] T 55d158eb3be242ad9e8b54580c108c6c P 3746ae9981e84ed2ae955a063c7f9cde [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:45:16.853538 30250 raft_consensus.cc:2258] T 55d158eb3be242ad9e8b54580c108c6c P 3746ae9981e84ed2ae955a063c7f9cde [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:45:16.854161 30249 ts_tablet_manager.cc:1905] T 55d158eb3be242ad9e8b54580c108c6c P 0717372a32fe47d9bc8a698607619b2e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:45:16.854075 29467 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53042:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:45:16.855170 29467 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:45:16.856386 30249 ts_tablet_manager.cc:1918] T 55d158eb3be242ad9e8b54580c108c6c P 0717372a32fe47d9bc8a698607619b2e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:45:16.856485 30249 log.cc:1196] T 55d158eb3be242ad9e8b54580c108c6c P 0717372a32fe47d9bc8a698607619b2e: Deleting WAL directory at /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/ts-1/wal/wals/55d158eb3be242ad9e8b54580c108c6c
I20260806 22:45:16.856817 30249 ts_tablet_manager.cc:1939] T 55d158eb3be242ad9e8b54580c108c6c P 0717372a32fe47d9bc8a698607619b2e: Deleting consensus metadata
I20260806 22:45:16.857594 29440 catalog_manager.cc:4841] TS 0717372a32fe47d9bc8a698607619b2e (127.18.45.194:33361): tablet 55d158eb3be242ad9e8b54580c108c6c (table test-table [id=12a37740cef5430e830a20c9e5f77b1f]) successfully deleted
I20260806 22:45:16.857764 30248 tablet_replica.cc:330] T 55d158eb3be242ad9e8b54580c108c6c P 8fe5cd5567524f4d928ba5b93ce819f6: stopping tablet replica
I20260806 22:45:16.857865 29673 tablet_service.cc:1467] Processing CreateTablet for tablet a4fc0386d65744b4806f311357ac79ec (DEFAULT_TABLE table=test-table [id=1a3ae0f06bcd4903a13c9b747ee4644c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:45:16.857928 30248 raft_consensus.cc:2229] T 55d158eb3be242ad9e8b54580c108c6c P 8fe5cd5567524f4d928ba5b93ce819f6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:45:16.858023 29673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4fc0386d65744b4806f311357ac79ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:45:16.858103 30248 raft_consensus.cc:2258] T 55d158eb3be242ad9e8b54580c108c6c P 8fe5cd5567524f4d928ba5b93ce819f6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:45:16.859328 29876 tablet_service.cc:1467] Processing CreateTablet for tablet a4fc0386d65744b4806f311357ac79ec (DEFAULT_TABLE table=test-table [id=1a3ae0f06bcd4903a13c9b747ee4644c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:45:16.859340 30079 tablet_service.cc:1467] Processing CreateTablet for tablet a4fc0386d65744b4806f311357ac79ec (DEFAULT_TABLE table=test-table [id=1a3ae0f06bcd4903a13c9b747ee4644c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:45:16.859472 29876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4fc0386d65744b4806f311357ac79ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:45:16.859473 30079 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4fc0386d65744b4806f311357ac79ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:45:16.859565 30250 ts_tablet_manager.cc:1905] T 55d158eb3be242ad9e8b54580c108c6c P 3746ae9981e84ed2ae955a063c7f9cde: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:45:16.859956 30215 tablet_bootstrap.cc:492] T a4fc0386d65744b4806f311357ac79ec P 3746ae9981e84ed2ae955a063c7f9cde: Bootstrap starting.
I20260806 22:45:16.860484 30215 tablet_bootstrap.cc:654] T a4fc0386d65744b4806f311357ac79ec P 3746ae9981e84ed2ae955a063c7f9cde: Neither blocks nor log segments found. Creating new log.
I20260806 22:45:16.861163 30218 tablet_bootstrap.cc:492] T a4fc0386d65744b4806f311357ac79ec P 0717372a32fe47d9bc8a698607619b2e: Bootstrap starting.
I20260806 22:45:16.861351 30214 tablet_bootstrap.cc:492] T a4fc0386d65744b4806f311357ac79ec P 8fe5cd5567524f4d928ba5b93ce819f6: Bootstrap starting.
I20260806 22:45:16.861373 30215 tablet_bootstrap.cc:492] T a4fc0386d65744b4806f311357ac79ec P 3746ae9981e84ed2ae955a063c7f9cde: No bootstrap required, opened a new log
I20260806 22:45:16.861425 30215 ts_tablet_manager.cc:1397] T a4fc0386d65744b4806f311357ac79ec P 3746ae9981e84ed2ae955a063c7f9cde: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:45:16.861615 30215 raft_consensus.cc:348] T a4fc0386d65744b4806f311357ac79ec P 3746ae9981e84ed2ae955a063c7f9cde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3746ae9981e84ed2ae955a063c7f9cde" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 35205 } } peers { permanent_uuid: "0717372a32fe47d9bc8a698607619b2e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33361 } } peers { permanent_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 37031 } }
I20260806 22:45:16.861666 30218 tablet_bootstrap.cc:654] T a4fc0386d65744b4806f311357ac79ec P 0717372a32fe47d9bc8a698607619b2e: Neither blocks nor log segments found. Creating new log.
I20260806 22:45:16.861698 30215 raft_consensus.cc:374] T a4fc0386d65744b4806f311357ac79ec P 3746ae9981e84ed2ae955a063c7f9cde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:45:16.861680 30248 ts_tablet_manager.cc:1905] T 55d158eb3be242ad9e8b54580c108c6c P 8fe5cd5567524f4d928ba5b93ce819f6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:45:16.861718 30215 raft_consensus.cc:729] T a4fc0386d65744b4806f311357ac79ec P 3746ae9981e84ed2ae955a063c7f9cde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3746ae9981e84ed2ae955a063c7f9cde, State: Initialized, Role: FOLLOWER
I20260806 22:45:16.861766 30215 consensus_queue.cc:260] T a4fc0386d65744b4806f311357ac79ec P 3746ae9981e84ed2ae955a063c7f9cde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3746ae9981e84ed2ae955a063c7f9cde" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 35205 } } peers { permanent_uuid: "0717372a32fe47d9bc8a698607619b2e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33361 } } peers { permanent_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 37031 } }
I20260806 22:45:16.861867 30214 tablet_bootstrap.cc:654] T a4fc0386d65744b4806f311357ac79ec P 8fe5cd5567524f4d928ba5b93ce819f6: Neither blocks nor log segments found. Creating new log.
I20260806 22:45:16.861889 30215 ts_tablet_manager.cc:1428] T a4fc0386d65744b4806f311357ac79ec P 3746ae9981e84ed2ae955a063c7f9cde: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 22:45:16.863525 30248 ts_tablet_manager.cc:1918] T 55d158eb3be242ad9e8b54580c108c6c P 8fe5cd5567524f4d928ba5b93ce819f6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:45:16.863562 30214 tablet_bootstrap.cc:492] T a4fc0386d65744b4806f311357ac79ec P 8fe5cd5567524f4d928ba5b93ce819f6: No bootstrap required, opened a new log
I20260806 22:45:16.863629 30248 log.cc:1196] T 55d158eb3be242ad9e8b54580c108c6c P 8fe5cd5567524f4d928ba5b93ce819f6: Deleting WAL directory at /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/ts-2/wal/wals/55d158eb3be242ad9e8b54580c108c6c
I20260806 22:45:16.863644 30214 ts_tablet_manager.cc:1397] T a4fc0386d65744b4806f311357ac79ec P 8fe5cd5567524f4d928ba5b93ce819f6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:45:16.863823 30218 tablet_bootstrap.cc:492] T a4fc0386d65744b4806f311357ac79ec P 0717372a32fe47d9bc8a698607619b2e: No bootstrap required, opened a new log
I20260806 22:45:16.863878 30218 ts_tablet_manager.cc:1397] T a4fc0386d65744b4806f311357ac79ec P 0717372a32fe47d9bc8a698607619b2e: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 22:45:16.863855 30214 raft_consensus.cc:348] T a4fc0386d65744b4806f311357ac79ec P 8fe5cd5567524f4d928ba5b93ce819f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3746ae9981e84ed2ae955a063c7f9cde" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 35205 } } peers { permanent_uuid: "0717372a32fe47d9bc8a698607619b2e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33361 } } peers { permanent_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 37031 } }
I20260806 22:45:16.863930 30214 raft_consensus.cc:374] T a4fc0386d65744b4806f311357ac79ec P 8fe5cd5567524f4d928ba5b93ce819f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:45:16.863991 30214 raft_consensus.cc:729] T a4fc0386d65744b4806f311357ac79ec P 8fe5cd5567524f4d928ba5b93ce819f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fe5cd5567524f4d928ba5b93ce819f6, State: Initialized, Role: FOLLOWER
I20260806 22:45:16.864010 30248 ts_tablet_manager.cc:1939] T 55d158eb3be242ad9e8b54580c108c6c P 8fe5cd5567524f4d928ba5b93ce819f6: Deleting consensus metadata
I20260806 22:45:16.864071 30214 consensus_queue.cc:260] T a4fc0386d65744b4806f311357ac79ec P 8fe5cd5567524f4d928ba5b93ce819f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3746ae9981e84ed2ae955a063c7f9cde" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 35205 } } peers { permanent_uuid: "0717372a32fe47d9bc8a698607619b2e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33361 } } peers { permanent_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 37031 } }
I20260806 22:45:16.864217 30214 ts_tablet_manager.cc:1428] T a4fc0386d65744b4806f311357ac79ec P 8fe5cd5567524f4d928ba5b93ce819f6: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 22:45:16.864349 30218 raft_consensus.cc:348] T a4fc0386d65744b4806f311357ac79ec P 0717372a32fe47d9bc8a698607619b2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3746ae9981e84ed2ae955a063c7f9cde" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 35205 } } peers { permanent_uuid: "0717372a32fe47d9bc8a698607619b2e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33361 } } peers { permanent_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 37031 } }
I20260806 22:45:16.864444 30218 raft_consensus.cc:374] T a4fc0386d65744b4806f311357ac79ec P 0717372a32fe47d9bc8a698607619b2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:45:16.864470 30218 raft_consensus.cc:729] T a4fc0386d65744b4806f311357ac79ec P 0717372a32fe47d9bc8a698607619b2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0717372a32fe47d9bc8a698607619b2e, State: Initialized, Role: FOLLOWER
I20260806 22:45:16.864526 30218 consensus_queue.cc:260] T a4fc0386d65744b4806f311357ac79ec P 0717372a32fe47d9bc8a698607619b2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3746ae9981e84ed2ae955a063c7f9cde" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 35205 } } peers { permanent_uuid: "0717372a32fe47d9bc8a698607619b2e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33361 } } peers { permanent_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 37031 } }
I20260806 22:45:16.864687 30218 ts_tablet_manager.cc:1428] T a4fc0386d65744b4806f311357ac79ec P 0717372a32fe47d9bc8a698607619b2e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 22:45:16.864914 29444 catalog_manager.cc:4841] TS 8fe5cd5567524f4d928ba5b93ce819f6 (127.18.45.195:37031): tablet 55d158eb3be242ad9e8b54580c108c6c (table test-table [id=12a37740cef5430e830a20c9e5f77b1f]) successfully deleted
I20260806 22:45:16.867334 30250 ts_tablet_manager.cc:1918] T 55d158eb3be242ad9e8b54580c108c6c P 3746ae9981e84ed2ae955a063c7f9cde: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:45:16.867456 30250 log.cc:1196] T 55d158eb3be242ad9e8b54580c108c6c P 3746ae9981e84ed2ae955a063c7f9cde: Deleting WAL directory at /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/ts-0/wal/wals/55d158eb3be242ad9e8b54580c108c6c
I20260806 22:45:16.867806 30250 ts_tablet_manager.cc:1939] T 55d158eb3be242ad9e8b54580c108c6c P 3746ae9981e84ed2ae955a063c7f9cde: Deleting consensus metadata
I20260806 22:45:16.868409 29440 catalog_manager.cc:4841] TS 3746ae9981e84ed2ae955a063c7f9cde (127.18.45.193:35205): tablet 55d158eb3be242ad9e8b54580c108c6c (table test-table [id=12a37740cef5430e830a20c9e5f77b1f]) successfully deleted
I20260806 22:45:17.174842 30225 raft_consensus.cc:482] T a4fc0386d65744b4806f311357ac79ec P 0717372a32fe47d9bc8a698607619b2e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:45:17.175061 30225 raft_consensus.cc:504] T a4fc0386d65744b4806f311357ac79ec P 0717372a32fe47d9bc8a698607619b2e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3746ae9981e84ed2ae955a063c7f9cde" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 35205 } } peers { permanent_uuid: "0717372a32fe47d9bc8a698607619b2e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33361 } } peers { permanent_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 37031 } }
I20260806 22:45:17.175545 30225 leader_election.cc:288] T a4fc0386d65744b4806f311357ac79ec P 0717372a32fe47d9bc8a698607619b2e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3746ae9981e84ed2ae955a063c7f9cde (127.18.45.193:35205), 8fe5cd5567524f4d928ba5b93ce819f6 (127.18.45.195:37031)
I20260806 22:45:17.179068 30210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:17.175918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:43355 (local address 127.18.45.195:37031)
0806 22:45:17.175986 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:17.175989 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:17.176076 (+ 87us) server_negotiation.cc:407] Connection header received
0806 22:45:17.176125 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:17.176127 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:17.176140 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:17.176208 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:45:17.176486 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:17.177027 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:17.178191 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:17.178604 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:17.178687 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:17.178700 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:17.178897 (+ 197us) server_negotiation.cc:299] Negotiation successful
0806 22:45:17.178946 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 22:45:17.179070 30213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:17.175881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:56107 (local address 127.18.45.193:35205)
0806 22:45:17.176000 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:17.176002 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:17.176168 (+ 166us) server_negotiation.cc:407] Connection header received
0806 22:45:17.176242 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:17.176243 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:17.176260 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:17.176315 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:45:17.176522 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:17.177090 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:17.178193 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:17.178609 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:17.178703 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:17.178718 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:17.178893 (+ 175us) server_negotiation.cc:299] Negotiation successful
0806 22:45:17.178955 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260806 22:45:17.179114 30261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:17.175786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.193:35205 (local address 127.18.45.194:56107)
0806 22:45:17.176069 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:17.176081 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:17.176162 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:17.176330 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:17.176331 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:17.176333 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:45:17.176413 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:17.176414 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:17.177183 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:17.177183 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:17.178068 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:17.178071 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:17.178723 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:17.178724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:17.178806 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:17.178810 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:45:17.178883 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 22:45:17.178953 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"mutex_wait_us":31,"thread_start_us":86,"threads_started":1}
I20260806 22:45:17.179127 30259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:17.175784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.195:37031 (local address 127.18.45.194:43355)
0806 22:45:17.175968 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:17.175982 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:17.176064 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:17.176233 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:17.176234 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:17.176236 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:45:17.176394 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:17.176397 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:17.177127 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:17.177128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:17.178071 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:17.178074 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:17.178722 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:17.178723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:17.178811 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:17.178814 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:45:17.178887 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 22:45:17.178938 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"mutex_wait_us":6,"thread_start_us":135,"threads_started":1}
I20260806 22:45:17.179577 30104 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4fc0386d65744b4806f311357ac79ec" candidate_uuid: "0717372a32fe47d9bc8a698607619b2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" is_pre_election: true
I20260806 22:45:17.179531 29693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4fc0386d65744b4806f311357ac79ec" candidate_uuid: "0717372a32fe47d9bc8a698607619b2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3746ae9981e84ed2ae955a063c7f9cde" is_pre_election: true
I20260806 22:45:17.179693 30104 raft_consensus.cc:2454] T a4fc0386d65744b4806f311357ac79ec P 8fe5cd5567524f4d928ba5b93ce819f6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0717372a32fe47d9bc8a698607619b2e in term 0.
I20260806 22:45:17.179738 29693 raft_consensus.cc:2454] T a4fc0386d65744b4806f311357ac79ec P 3746ae9981e84ed2ae955a063c7f9cde [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0717372a32fe47d9bc8a698607619b2e in term 0.
I20260806 22:45:17.179936 29807 leader_election.cc:302] T a4fc0386d65744b4806f311357ac79ec P 0717372a32fe47d9bc8a698607619b2e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0717372a32fe47d9bc8a698607619b2e, 8fe5cd5567524f4d928ba5b93ce819f6; no voters:
I20260806 22:45:17.180110 30225 raft_consensus.cc:2781] T a4fc0386d65744b4806f311357ac79ec P 0717372a32fe47d9bc8a698607619b2e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:45:17.180168 30225 raft_consensus.cc:482] T a4fc0386d65744b4806f311357ac79ec P 0717372a32fe47d9bc8a698607619b2e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:45:17.180191 30225 raft_consensus.cc:3037] T a4fc0386d65744b4806f311357ac79ec P 0717372a32fe47d9bc8a698607619b2e [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:45:17.181082 30225 raft_consensus.cc:504] T a4fc0386d65744b4806f311357ac79ec P 0717372a32fe47d9bc8a698607619b2e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3746ae9981e84ed2ae955a063c7f9cde" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 35205 } } peers { permanent_uuid: "0717372a32fe47d9bc8a698607619b2e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33361 } } peers { permanent_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 37031 } }
I20260806 22:45:17.181257 30225 leader_election.cc:288] T a4fc0386d65744b4806f311357ac79ec P 0717372a32fe47d9bc8a698607619b2e [CANDIDATE]: Term 1 election: Requested vote from peers 3746ae9981e84ed2ae955a063c7f9cde (127.18.45.193:35205), 8fe5cd5567524f4d928ba5b93ce819f6 (127.18.45.195:37031)
I20260806 22:45:17.181447 29693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4fc0386d65744b4806f311357ac79ec" candidate_uuid: "0717372a32fe47d9bc8a698607619b2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3746ae9981e84ed2ae955a063c7f9cde"
I20260806 22:45:17.181442 30104 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4fc0386d65744b4806f311357ac79ec" candidate_uuid: "0717372a32fe47d9bc8a698607619b2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe5cd5567524f4d928ba5b93ce819f6"
I20260806 22:45:17.181535 29693 raft_consensus.cc:3037] T a4fc0386d65744b4806f311357ac79ec P 3746ae9981e84ed2ae955a063c7f9cde [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:45:17.181556 30104 raft_consensus.cc:3037] T a4fc0386d65744b4806f311357ac79ec P 8fe5cd5567524f4d928ba5b93ce819f6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:45:17.182359 29693 raft_consensus.cc:2454] T a4fc0386d65744b4806f311357ac79ec P 3746ae9981e84ed2ae955a063c7f9cde [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0717372a32fe47d9bc8a698607619b2e in term 1.
I20260806 22:45:17.182402 30104 raft_consensus.cc:2454] T a4fc0386d65744b4806f311357ac79ec P 8fe5cd5567524f4d928ba5b93ce819f6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0717372a32fe47d9bc8a698607619b2e in term 1.
I20260806 22:45:17.182588 29802 leader_election.cc:302] T a4fc0386d65744b4806f311357ac79ec P 0717372a32fe47d9bc8a698607619b2e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0717372a32fe47d9bc8a698607619b2e, 3746ae9981e84ed2ae955a063c7f9cde; no voters:
I20260806 22:45:17.182718 30225 raft_consensus.cc:2781] T a4fc0386d65744b4806f311357ac79ec P 0717372a32fe47d9bc8a698607619b2e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:45:17.182896 30225 raft_consensus.cc:686] T a4fc0386d65744b4806f311357ac79ec P 0717372a32fe47d9bc8a698607619b2e [term 1 LEADER]: Becoming Leader. State: Replica: 0717372a32fe47d9bc8a698607619b2e, State: Running, Role: LEADER
I20260806 22:45:17.183002 30225 consensus_queue.cc:237] T a4fc0386d65744b4806f311357ac79ec P 0717372a32fe47d9bc8a698607619b2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3746ae9981e84ed2ae955a063c7f9cde" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 35205 } } peers { permanent_uuid: "0717372a32fe47d9bc8a698607619b2e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33361 } } peers { permanent_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 37031 } }
I20260806 22:45:17.183926 29466 catalog_manager.cc:5495] T a4fc0386d65744b4806f311357ac79ec P 0717372a32fe47d9bc8a698607619b2e reported cstate change: term changed from 0 to 1, leader changed from <none> to 0717372a32fe47d9bc8a698607619b2e (127.18.45.194). New cstate: current_term: 1 leader_uuid: "0717372a32fe47d9bc8a698607619b2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3746ae9981e84ed2ae955a063c7f9cde" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 35205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0717372a32fe47d9bc8a698607619b2e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33361 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 37031 } health_report { overall_health: UNKNOWN } } }
Aug 06 22:45:17 dist-test-slave-h3ft krb5kdc[29399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056315, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.45.254@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM
I20260806 22:45:17.248721 30270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:17.240067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:41637 (local address 127.0.0.1:53056)
0806 22:45:17.240264 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:17.240279 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:17.240389 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:17.240598 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:17.240599 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:17.241027 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:17.241107 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:17.241111 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:17.241875 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:17.241875 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:17.242510 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:17.242512 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:17.242881 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:17.242881 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:17.242930 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:17.243612 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:17.243618 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:17.245391 (+ 1773us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:45:17.247402 (+ 2011us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:17.247409 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:17.247412 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:17.247657 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:17.247851 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:17.247851 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:17.247852 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:17.247927 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:17.248298 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:45:17.248306 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:45:17.248359 (+ 53us) client_negotiation.cc:769] Sending connection context
0806 22:45:17.248437 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 22:45:17.248567 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":90,"threads_started":1}
I20260806 22:45:17.248842 30271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:17.240153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53056 (local address 127.18.45.254:41637)
0806 22:45:17.240367 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:17.240369 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:17.240381 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:45:17.240457 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:17.240458 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:17.240470 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:17.240549 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:17.241198 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:17.241762 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:17.242602 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:17.242803 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:17.242869 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:17.245499 (+ 2630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:17.245501 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:17.245501 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:17.245509 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:17.247328 (+ 1819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:17.247724 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:17.247725 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:17.247726 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:17.247789 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:17.248059 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:17.248060 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:17.248060 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:17.248205 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:17.248303 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:17.248480 (+ 177us) server_negotiation.cc:299] Negotiation successful
0806 22:45:17.248624 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":63,"server-negotiator.queue_time_us":188,"thread_start_us":147,"threads_started":1}
I20260806 22:45:17.251811 29466 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:53056:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 22:45:17.252159 29466 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:45:17.254138 29876 tablet_service.cc:1467] Processing CreateTablet for tablet d841b7e77b024ebcb17b9f7d06ea8e96 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5b189a395a574915b8fb41e09b4352ee]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:45:17.254150 29673 tablet_service.cc:1467] Processing CreateTablet for tablet d841b7e77b024ebcb17b9f7d06ea8e96 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5b189a395a574915b8fb41e09b4352ee]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:45:17.254172 30079 tablet_service.cc:1467] Processing CreateTablet for tablet d841b7e77b024ebcb17b9f7d06ea8e96 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5b189a395a574915b8fb41e09b4352ee]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 22:45:17.254349 30079 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d841b7e77b024ebcb17b9f7d06ea8e96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:45:17.254359 29876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d841b7e77b024ebcb17b9f7d06ea8e96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:45:17.254371 29673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d841b7e77b024ebcb17b9f7d06ea8e96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:45:17.256201 30214 tablet_bootstrap.cc:492] T d841b7e77b024ebcb17b9f7d06ea8e96 P 8fe5cd5567524f4d928ba5b93ce819f6: Bootstrap starting.
I20260806 22:45:17.256335 30218 tablet_bootstrap.cc:492] T d841b7e77b024ebcb17b9f7d06ea8e96 P 0717372a32fe47d9bc8a698607619b2e: Bootstrap starting.
I20260806 22:45:17.256732 30214 tablet_bootstrap.cc:654] T d841b7e77b024ebcb17b9f7d06ea8e96 P 8fe5cd5567524f4d928ba5b93ce819f6: Neither blocks nor log segments found. Creating new log.
I20260806 22:45:17.256803 30218 tablet_bootstrap.cc:654] T d841b7e77b024ebcb17b9f7d06ea8e96 P 0717372a32fe47d9bc8a698607619b2e: Neither blocks nor log segments found. Creating new log.
I20260806 22:45:17.257201 30215 tablet_bootstrap.cc:492] T d841b7e77b024ebcb17b9f7d06ea8e96 P 3746ae9981e84ed2ae955a063c7f9cde: Bootstrap starting.
I20260806 22:45:17.257622 30215 tablet_bootstrap.cc:654] T d841b7e77b024ebcb17b9f7d06ea8e96 P 3746ae9981e84ed2ae955a063c7f9cde: Neither blocks nor log segments found. Creating new log.
I20260806 22:45:17.257670 30218 tablet_bootstrap.cc:492] T d841b7e77b024ebcb17b9f7d06ea8e96 P 0717372a32fe47d9bc8a698607619b2e: No bootstrap required, opened a new log
I20260806 22:45:17.257730 30218 ts_tablet_manager.cc:1397] T d841b7e77b024ebcb17b9f7d06ea8e96 P 0717372a32fe47d9bc8a698607619b2e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:45:17.257936 30218 raft_consensus.cc:348] T d841b7e77b024ebcb17b9f7d06ea8e96 P 0717372a32fe47d9bc8a698607619b2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3746ae9981e84ed2ae955a063c7f9cde" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 35205 } } peers { permanent_uuid: "0717372a32fe47d9bc8a698607619b2e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33361 } } peers { permanent_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 37031 } }
I20260806 22:45:17.258045 30218 raft_consensus.cc:374] T d841b7e77b024ebcb17b9f7d06ea8e96 P 0717372a32fe47d9bc8a698607619b2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:45:17.258080 30218 raft_consensus.cc:729] T d841b7e77b024ebcb17b9f7d06ea8e96 P 0717372a32fe47d9bc8a698607619b2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0717372a32fe47d9bc8a698607619b2e, State: Initialized, Role: FOLLOWER
I20260806 22:45:17.258131 30218 consensus_queue.cc:260] T d841b7e77b024ebcb17b9f7d06ea8e96 P 0717372a32fe47d9bc8a698607619b2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3746ae9981e84ed2ae955a063c7f9cde" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 35205 } } peers { permanent_uuid: "0717372a32fe47d9bc8a698607619b2e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33361 } } peers { permanent_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 37031 } }
I20260806 22:45:17.258250 30218 ts_tablet_manager.cc:1428] T d841b7e77b024ebcb17b9f7d06ea8e96 P 0717372a32fe47d9bc8a698607619b2e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:17.258250 30214 tablet_bootstrap.cc:492] T d841b7e77b024ebcb17b9f7d06ea8e96 P 8fe5cd5567524f4d928ba5b93ce819f6: No bootstrap required, opened a new log
I20260806 22:45:17.258313 30214 ts_tablet_manager.cc:1397] T d841b7e77b024ebcb17b9f7d06ea8e96 P 8fe5cd5567524f4d928ba5b93ce819f6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:45:17.258350 30225 tablet_replica.cc:439] T d841b7e77b024ebcb17b9f7d06ea8e96 P 0717372a32fe47d9bc8a698607619b2e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3746ae9981e84ed2ae955a063c7f9cde" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 35205 } } peers { permanent_uuid: "0717372a32fe47d9bc8a698607619b2e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33361 } } peers { permanent_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 37031 } } }
I20260806 22:45:17.258482 30225 tablet_replica.cc:442] T d841b7e77b024ebcb17b9f7d06ea8e96 P 0717372a32fe47d9bc8a698607619b2e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:45:17.258533 30214 raft_consensus.cc:348] T d841b7e77b024ebcb17b9f7d06ea8e96 P 8fe5cd5567524f4d928ba5b93ce819f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3746ae9981e84ed2ae955a063c7f9cde" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 35205 } } peers { permanent_uuid: "0717372a32fe47d9bc8a698607619b2e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33361 } } peers { permanent_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 37031 } }
I20260806 22:45:17.258615 30214 raft_consensus.cc:374] T d841b7e77b024ebcb17b9f7d06ea8e96 P 8fe5cd5567524f4d928ba5b93ce819f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:45:17.258635 30214 raft_consensus.cc:729] T d841b7e77b024ebcb17b9f7d06ea8e96 P 8fe5cd5567524f4d928ba5b93ce819f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fe5cd5567524f4d928ba5b93ce819f6, State: Initialized, Role: FOLLOWER
I20260806 22:45:17.258690 30214 consensus_queue.cc:260] T d841b7e77b024ebcb17b9f7d06ea8e96 P 8fe5cd5567524f4d928ba5b93ce819f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3746ae9981e84ed2ae955a063c7f9cde" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 35205 } } peers { permanent_uuid: "0717372a32fe47d9bc8a698607619b2e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33361 } } peers { permanent_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 37031 } }
I20260806 22:45:17.258792 30214 ts_tablet_manager.cc:1428] T d841b7e77b024ebcb17b9f7d06ea8e96 P 8fe5cd5567524f4d928ba5b93ce819f6: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:17.258865 30222 tablet_replica.cc:439] T d841b7e77b024ebcb17b9f7d06ea8e96 P 8fe5cd5567524f4d928ba5b93ce819f6: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3746ae9981e84ed2ae955a063c7f9cde" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 35205 } } peers { permanent_uuid: "0717372a32fe47d9bc8a698607619b2e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33361 } } peers { permanent_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 37031 } } }
I20260806 22:45:17.258944 30222 tablet_replica.cc:442] T d841b7e77b024ebcb17b9f7d06ea8e96 P 8fe5cd5567524f4d928ba5b93ce819f6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:45:17.259120 30215 tablet_bootstrap.cc:492] T d841b7e77b024ebcb17b9f7d06ea8e96 P 3746ae9981e84ed2ae955a063c7f9cde: No bootstrap required, opened a new log
I20260806 22:45:17.259184 30215 ts_tablet_manager.cc:1397] T d841b7e77b024ebcb17b9f7d06ea8e96 P 3746ae9981e84ed2ae955a063c7f9cde: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:45:17.259325 30215 raft_consensus.cc:348] T d841b7e77b024ebcb17b9f7d06ea8e96 P 3746ae9981e84ed2ae955a063c7f9cde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3746ae9981e84ed2ae955a063c7f9cde" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 35205 } } peers { permanent_uuid: "0717372a32fe47d9bc8a698607619b2e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33361 } } peers { permanent_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 37031 } }
I20260806 22:45:17.259408 30215 raft_consensus.cc:374] T d841b7e77b024ebcb17b9f7d06ea8e96 P 3746ae9981e84ed2ae955a063c7f9cde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:45:17.259430 30215 raft_consensus.cc:729] T d841b7e77b024ebcb17b9f7d06ea8e96 P 3746ae9981e84ed2ae955a063c7f9cde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3746ae9981e84ed2ae955a063c7f9cde, State: Initialized, Role: FOLLOWER
I20260806 22:45:17.259486 30215 consensus_queue.cc:260] T d841b7e77b024ebcb17b9f7d06ea8e96 P 3746ae9981e84ed2ae955a063c7f9cde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3746ae9981e84ed2ae955a063c7f9cde" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 35205 } } peers { permanent_uuid: "0717372a32fe47d9bc8a698607619b2e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33361 } } peers { permanent_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 37031 } }
I20260806 22:45:17.259616 30215 ts_tablet_manager.cc:1428] T d841b7e77b024ebcb17b9f7d06ea8e96 P 3746ae9981e84ed2ae955a063c7f9cde: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:17.259637 30223 tablet_replica.cc:439] T d841b7e77b024ebcb17b9f7d06ea8e96 P 3746ae9981e84ed2ae955a063c7f9cde: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3746ae9981e84ed2ae955a063c7f9cde" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 35205 } } peers { permanent_uuid: "0717372a32fe47d9bc8a698607619b2e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33361 } } peers { permanent_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 37031 } } }
I20260806 22:45:17.259739 30223 tablet_replica.cc:442] T d841b7e77b024ebcb17b9f7d06ea8e96 P 3746ae9981e84ed2ae955a063c7f9cde: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:45:17.286531 30225 raft_consensus.cc:482] T d841b7e77b024ebcb17b9f7d06ea8e96 P 0717372a32fe47d9bc8a698607619b2e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:45:17.286674 30225 raft_consensus.cc:504] T d841b7e77b024ebcb17b9f7d06ea8e96 P 0717372a32fe47d9bc8a698607619b2e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3746ae9981e84ed2ae955a063c7f9cde" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 35205 } } peers { permanent_uuid: "0717372a32fe47d9bc8a698607619b2e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33361 } } peers { permanent_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 37031 } }
I20260806 22:45:17.286888 30225 leader_election.cc:288] T d841b7e77b024ebcb17b9f7d06ea8e96 P 0717372a32fe47d9bc8a698607619b2e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3746ae9981e84ed2ae955a063c7f9cde (127.18.45.193:35205), 8fe5cd5567524f4d928ba5b93ce819f6 (127.18.45.195:37031)
I20260806 22:45:17.287158 29693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d841b7e77b024ebcb17b9f7d06ea8e96" candidate_uuid: "0717372a32fe47d9bc8a698607619b2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3746ae9981e84ed2ae955a063c7f9cde" is_pre_election: true
I20260806 22:45:17.287237 30104 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d841b7e77b024ebcb17b9f7d06ea8e96" candidate_uuid: "0717372a32fe47d9bc8a698607619b2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" is_pre_election: true
I20260806 22:45:17.287326 29693 raft_consensus.cc:2454] T d841b7e77b024ebcb17b9f7d06ea8e96 P 3746ae9981e84ed2ae955a063c7f9cde [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0717372a32fe47d9bc8a698607619b2e in term 0.
I20260806 22:45:17.287353 30104 raft_consensus.cc:2454] T d841b7e77b024ebcb17b9f7d06ea8e96 P 8fe5cd5567524f4d928ba5b93ce819f6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0717372a32fe47d9bc8a698607619b2e in term 0.
I20260806 22:45:17.287514 29802 leader_election.cc:302] T d841b7e77b024ebcb17b9f7d06ea8e96 P 0717372a32fe47d9bc8a698607619b2e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0717372a32fe47d9bc8a698607619b2e, 3746ae9981e84ed2ae955a063c7f9cde; no voters:
I20260806 22:45:17.287640 30225 raft_consensus.cc:2781] T d841b7e77b024ebcb17b9f7d06ea8e96 P 0717372a32fe47d9bc8a698607619b2e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:45:17.287690 30225 raft_consensus.cc:482] T d841b7e77b024ebcb17b9f7d06ea8e96 P 0717372a32fe47d9bc8a698607619b2e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:45:17.287706 30225 raft_consensus.cc:3037] T d841b7e77b024ebcb17b9f7d06ea8e96 P 0717372a32fe47d9bc8a698607619b2e [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:45:17.288573 30225 raft_consensus.cc:504] T d841b7e77b024ebcb17b9f7d06ea8e96 P 0717372a32fe47d9bc8a698607619b2e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3746ae9981e84ed2ae955a063c7f9cde" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 35205 } } peers { permanent_uuid: "0717372a32fe47d9bc8a698607619b2e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33361 } } peers { permanent_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 37031 } }
I20260806 22:45:17.288744 30225 leader_election.cc:288] T d841b7e77b024ebcb17b9f7d06ea8e96 P 0717372a32fe47d9bc8a698607619b2e [CANDIDATE]: Term 1 election: Requested vote from peers 3746ae9981e84ed2ae955a063c7f9cde (127.18.45.193:35205), 8fe5cd5567524f4d928ba5b93ce819f6 (127.18.45.195:37031)
I20260806 22:45:17.288911 29693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d841b7e77b024ebcb17b9f7d06ea8e96" candidate_uuid: "0717372a32fe47d9bc8a698607619b2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3746ae9981e84ed2ae955a063c7f9cde"
I20260806 22:45:17.289000 29693 raft_consensus.cc:3037] T d841b7e77b024ebcb17b9f7d06ea8e96 P 3746ae9981e84ed2ae955a063c7f9cde [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:45:17.288950 30104 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d841b7e77b024ebcb17b9f7d06ea8e96" candidate_uuid: "0717372a32fe47d9bc8a698607619b2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe5cd5567524f4d928ba5b93ce819f6"
I20260806 22:45:17.289090 30104 raft_consensus.cc:3037] T d841b7e77b024ebcb17b9f7d06ea8e96 P 8fe5cd5567524f4d928ba5b93ce819f6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:45:17.289825 29693 raft_consensus.cc:2454] T d841b7e77b024ebcb17b9f7d06ea8e96 P 3746ae9981e84ed2ae955a063c7f9cde [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0717372a32fe47d9bc8a698607619b2e in term 1.
I20260806 22:45:17.290030 30104 raft_consensus.cc:2454] T d841b7e77b024ebcb17b9f7d06ea8e96 P 8fe5cd5567524f4d928ba5b93ce819f6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0717372a32fe47d9bc8a698607619b2e in term 1.
I20260806 22:45:17.290036 29802 leader_election.cc:302] T d841b7e77b024ebcb17b9f7d06ea8e96 P 0717372a32fe47d9bc8a698607619b2e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0717372a32fe47d9bc8a698607619b2e, 3746ae9981e84ed2ae955a063c7f9cde; no voters:
I20260806 22:45:17.290225 30225 raft_consensus.cc:2781] T d841b7e77b024ebcb17b9f7d06ea8e96 P 0717372a32fe47d9bc8a698607619b2e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:45:17.290284 30225 raft_consensus.cc:686] T d841b7e77b024ebcb17b9f7d06ea8e96 P 0717372a32fe47d9bc8a698607619b2e [term 1 LEADER]: Becoming Leader. State: Replica: 0717372a32fe47d9bc8a698607619b2e, State: Running, Role: LEADER
I20260806 22:45:17.290378 30225 consensus_queue.cc:237] T d841b7e77b024ebcb17b9f7d06ea8e96 P 0717372a32fe47d9bc8a698607619b2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3746ae9981e84ed2ae955a063c7f9cde" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 35205 } } peers { permanent_uuid: "0717372a32fe47d9bc8a698607619b2e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33361 } } peers { permanent_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 37031 } }
I20260806 22:45:17.290730 30262 tablet_replica.cc:439] T d841b7e77b024ebcb17b9f7d06ea8e96 P 0717372a32fe47d9bc8a698607619b2e: TxnStatusTablet state changed. Reason: New leader 0717372a32fe47d9bc8a698607619b2e. Latest consensus state: current_term: 1 leader_uuid: "0717372a32fe47d9bc8a698607619b2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3746ae9981e84ed2ae955a063c7f9cde" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 35205 } } peers { permanent_uuid: "0717372a32fe47d9bc8a698607619b2e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33361 } } peers { permanent_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 37031 } } }
I20260806 22:45:17.290832 30262 tablet_replica.cc:442] T d841b7e77b024ebcb17b9f7d06ea8e96 P 0717372a32fe47d9bc8a698607619b2e: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:45:17.291154 30278 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 22:45:17.291178 29466 catalog_manager.cc:5495] T d841b7e77b024ebcb17b9f7d06ea8e96 P 0717372a32fe47d9bc8a698607619b2e reported cstate change: term changed from 0 to 1, leader changed from <none> to 0717372a32fe47d9bc8a698607619b2e (127.18.45.194). New cstate: current_term: 1 leader_uuid: "0717372a32fe47d9bc8a698607619b2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3746ae9981e84ed2ae955a063c7f9cde" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 35205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0717372a32fe47d9bc8a698607619b2e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33361 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 37031 } health_report { overall_health: UNKNOWN } } }
I20260806 22:45:17.291340 30278 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 22:45:17.298785 30270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:17.295234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.194:33361 (local address 127.0.0.1:46340)
0806 22:45:17.295342 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:17.295354 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:17.295397 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:17.295731 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:17.295733 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:17.295735 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:45:17.295804 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:17.295807 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:17.296762 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:17.296763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:17.297569 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:17.297571 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:17.298074 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:17.298076 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:17.298167 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:17.298187 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:45:17.298546 (+ 359us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:45:17.298560 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 22:45:17.298620 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 22:45:17.298697 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260806 22:45:17.298791 30283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:17.295302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46340 (local address 127.18.45.194:33361)
0806 22:45:17.295546 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:17.295549 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:17.295557 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:45:17.295594 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:17.295595 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:17.295614 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:17.295706 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:45:17.295902 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:17.296658 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:17.297663 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:17.297976 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:17.298066 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:17.298309 (+ 243us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:45:17.298453 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:45:17.298551 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:17.298628 (+ 77us) server_negotiation.cc:299] Negotiation successful
0806 22:45:17.298656 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":120,"threads_started":1}
I20260806 22:45:17.300283 30104 raft_consensus.cc:1261] T d841b7e77b024ebcb17b9f7d06ea8e96 P 8fe5cd5567524f4d928ba5b93ce819f6 [term 1 FOLLOWER]: Refusing update from remote peer 0717372a32fe47d9bc8a698607619b2e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:45:17.300402 29693 raft_consensus.cc:1261] T d841b7e77b024ebcb17b9f7d06ea8e96 P 3746ae9981e84ed2ae955a063c7f9cde [term 1 FOLLOWER]: Refusing update from remote peer 0717372a32fe47d9bc8a698607619b2e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:45:17.300625 30262 consensus_queue.cc:1035] T d841b7e77b024ebcb17b9f7d06ea8e96 P 0717372a32fe47d9bc8a698607619b2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 37031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:45:17.300773 30225 consensus_queue.cc:1035] T d841b7e77b024ebcb17b9f7d06ea8e96 P 0717372a32fe47d9bc8a698607619b2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3746ae9981e84ed2ae955a063c7f9cde" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 35205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:45:17.301574 30222 tablet_replica.cc:439] T d841b7e77b024ebcb17b9f7d06ea8e96 P 8fe5cd5567524f4d928ba5b93ce819f6: TxnStatusTablet state changed. Reason: New leader 0717372a32fe47d9bc8a698607619b2e. Latest consensus state: current_term: 1 leader_uuid: "0717372a32fe47d9bc8a698607619b2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3746ae9981e84ed2ae955a063c7f9cde" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 35205 } } peers { permanent_uuid: "0717372a32fe47d9bc8a698607619b2e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33361 } } peers { permanent_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 37031 } } }
I20260806 22:45:17.301743 30222 tablet_replica.cc:442] T d841b7e77b024ebcb17b9f7d06ea8e96 P 8fe5cd5567524f4d928ba5b93ce819f6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:45:17.301781 30223 tablet_replica.cc:439] T d841b7e77b024ebcb17b9f7d06ea8e96 P 3746ae9981e84ed2ae955a063c7f9cde: TxnStatusTablet state changed. Reason: New leader 0717372a32fe47d9bc8a698607619b2e. Latest consensus state: current_term: 1 leader_uuid: "0717372a32fe47d9bc8a698607619b2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3746ae9981e84ed2ae955a063c7f9cde" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 35205 } } peers { permanent_uuid: "0717372a32fe47d9bc8a698607619b2e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33361 } } peers { permanent_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 37031 } } }
I20260806 22:45:17.301882 30223 tablet_replica.cc:442] T d841b7e77b024ebcb17b9f7d06ea8e96 P 3746ae9981e84ed2ae955a063c7f9cde: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:45:17.303323 30223 tablet_replica.cc:439] T d841b7e77b024ebcb17b9f7d06ea8e96 P 3746ae9981e84ed2ae955a063c7f9cde: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0717372a32fe47d9bc8a698607619b2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3746ae9981e84ed2ae955a063c7f9cde" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 35205 } } peers { permanent_uuid: "0717372a32fe47d9bc8a698607619b2e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33361 } } peers { permanent_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 37031 } } }
I20260806 22:45:17.303414 30223 tablet_replica.cc:442] T d841b7e77b024ebcb17b9f7d06ea8e96 P 3746ae9981e84ed2ae955a063c7f9cde: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:45:17.303756 30262 tablet_replica.cc:439] T d841b7e77b024ebcb17b9f7d06ea8e96 P 0717372a32fe47d9bc8a698607619b2e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0717372a32fe47d9bc8a698607619b2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3746ae9981e84ed2ae955a063c7f9cde" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 35205 } } peers { permanent_uuid: "0717372a32fe47d9bc8a698607619b2e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33361 } } peers { permanent_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 37031 } } }
I20260806 22:45:17.303856 30262 tablet_replica.cc:442] T d841b7e77b024ebcb17b9f7d06ea8e96 P 0717372a32fe47d9bc8a698607619b2e: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:45:17.303810 30222 tablet_replica.cc:439] T d841b7e77b024ebcb17b9f7d06ea8e96 P 8fe5cd5567524f4d928ba5b93ce819f6: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0717372a32fe47d9bc8a698607619b2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3746ae9981e84ed2ae955a063c7f9cde" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 35205 } } peers { permanent_uuid: "0717372a32fe47d9bc8a698607619b2e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33361 } } peers { permanent_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 37031 } } }
I20260806 22:45:17.303911 30222 tablet_replica.cc:442] T d841b7e77b024ebcb17b9f7d06ea8e96 P 8fe5cd5567524f4d928ba5b93ce819f6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 22:45:17.303963 30232 mvcc.cc:205] Tried to move back new op lower bound from 7315686675659440128 to 7315686675621978112. Current Snapshot: MvccSnapshot[applied={T|T < 7315686675659440128}]
I20260806 22:45:17.302930 30225 tablet_replica.cc:439] T d841b7e77b024ebcb17b9f7d06ea8e96 P 0717372a32fe47d9bc8a698607619b2e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0717372a32fe47d9bc8a698607619b2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3746ae9981e84ed2ae955a063c7f9cde" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 35205 } } peers { permanent_uuid: "0717372a32fe47d9bc8a698607619b2e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33361 } } peers { permanent_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 37031 } } }
I20260806 22:45:17.304584 30225 tablet_replica.cc:442] T d841b7e77b024ebcb17b9f7d06ea8e96 P 0717372a32fe47d9bc8a698607619b2e: This TxnStatusTablet replica's current role is: LEADER
I20260806 22:45:17.309345 30283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:17.306453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46350 (local address 127.18.45.194:33361)
0806 22:45:17.306515 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:17.306517 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:17.306529 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:45:17.306610 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:17.306611 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:17.306624 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:17.306674 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:45:17.306945 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:17.307591 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:17.308486 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:17.308757 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:17.308844 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:17.309035 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:45:17.309077 (+ 42us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:45:17.309154 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:17.309225 (+ 71us) server_negotiation.cc:299] Negotiation successful
0806 22:45:17.309252 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 22:45:17.315495 30283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:17.312610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:54351 (local address 127.18.45.194:33361)
0806 22:45:17.312661 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:17.312663 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:17.312858 (+ 195us) server_negotiation.cc:407] Connection header received
0806 22:45:17.312924 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:17.312925 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:17.312936 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:17.312998 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:45:17.313190 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:17.313711 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:17.314636 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:17.314852 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:17.314931 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:17.315189 (+ 258us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:45:17.315227 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:45:17.315280 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:17.315342 (+ 62us) server_negotiation.cc:299] Negotiation successful
0806 22:45:17.315385 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":15,"server-negotiator.queue_time_us":21}
I20260806 22:45:17.315521 30293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:17.312556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.194:33361 (local address 127.18.45.194:54351)
0806 22:45:17.312768 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:17.312780 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:17.312858 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:17.313001 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:17.313001 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:17.313003 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:45:17.313111 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:17.313113 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:17.313797 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:17.313798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:17.314525 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:17.314527 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:17.314933 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:17.314934 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:17.315052 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:17.315073 (+ 21us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:45:17.315279 (+ 206us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:45:17.315280 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 22:45:17.315338 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 22:45:17.315405 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"mutex_wait_us":4,"thread_start_us":145,"threads_started":1}
I20260806 22:45:17.318305 30104 raft_consensus.cc:1261] T a4fc0386d65744b4806f311357ac79ec P 8fe5cd5567524f4d928ba5b93ce819f6 [term 1 FOLLOWER]: Refusing update from remote peer 0717372a32fe47d9bc8a698607619b2e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:45:17.318512 29693 raft_consensus.cc:1261] T a4fc0386d65744b4806f311357ac79ec P 3746ae9981e84ed2ae955a063c7f9cde [term 1 FOLLOWER]: Refusing update from remote peer 0717372a32fe47d9bc8a698607619b2e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:45:17.318656 30287 consensus_queue.cc:1035] T a4fc0386d65744b4806f311357ac79ec P 0717372a32fe47d9bc8a698607619b2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 37031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:45:17.318856 30288 consensus_queue.cc:1035] T a4fc0386d65744b4806f311357ac79ec P 0717372a32fe47d9bc8a698607619b2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3746ae9981e84ed2ae955a063c7f9cde" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 35205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:45:17.335870 29466 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53042:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:45:17.336022 29466 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53042:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:45:17.337047 29466 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f3e29cbdebbb43999c5d7cf6c549f7b8: Sending DeleteTablet for 3 replicas of tablet a4fc0386d65744b4806f311357ac79ec
I20260806 22:45:17.337607 30079 tablet_service.cc:1514] Processing DeleteTablet for tablet a4fc0386d65744b4806f311357ac79ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:45:17 UTC) from {username='oryx', principal='oryx/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:36598
I20260806 22:45:17.337661 29873 tablet_service.cc:1514] Processing DeleteTablet for tablet a4fc0386d65744b4806f311357ac79ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:45:17 UTC) from {username='oryx', principal='oryx/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:46334
I20260806 22:45:17.337778 30248 tablet_replica.cc:330] T a4fc0386d65744b4806f311357ac79ec P 8fe5cd5567524f4d928ba5b93ce819f6: stopping tablet replica
I20260806 22:45:17.337792 30249 tablet_replica.cc:330] T a4fc0386d65744b4806f311357ac79ec P 0717372a32fe47d9bc8a698607619b2e: stopping tablet replica
I20260806 22:45:17.337795 29673 tablet_service.cc:1514] Processing DeleteTablet for tablet a4fc0386d65744b4806f311357ac79ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:45:17 UTC) from {username='oryx', principal='oryx/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:54092
I20260806 22:45:17.337864 30249 raft_consensus.cc:2229] T a4fc0386d65744b4806f311357ac79ec P 0717372a32fe47d9bc8a698607619b2e [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:45:17.337868 30248 raft_consensus.cc:2229] T a4fc0386d65744b4806f311357ac79ec P 8fe5cd5567524f4d928ba5b93ce819f6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:45:17.337939 30248 raft_consensus.cc:2258] T a4fc0386d65744b4806f311357ac79ec P 8fe5cd5567524f4d928ba5b93ce819f6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:45:17.338020 30249 raft_consensus.cc:2258] T a4fc0386d65744b4806f311357ac79ec P 0717372a32fe47d9bc8a698607619b2e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:45:17.338049 30250 tablet_replica.cc:330] T a4fc0386d65744b4806f311357ac79ec P 3746ae9981e84ed2ae955a063c7f9cde: stopping tablet replica
I20260806 22:45:17.338138 30250 raft_consensus.cc:2229] T a4fc0386d65744b4806f311357ac79ec P 3746ae9981e84ed2ae955a063c7f9cde [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:45:17.338212 30250 raft_consensus.cc:2258] T a4fc0386d65744b4806f311357ac79ec P 3746ae9981e84ed2ae955a063c7f9cde [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:45:17.338485 30248 ts_tablet_manager.cc:1905] T a4fc0386d65744b4806f311357ac79ec P 8fe5cd5567524f4d928ba5b93ce819f6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:45:17.339638 30249 ts_tablet_manager.cc:1905] T a4fc0386d65744b4806f311357ac79ec P 0717372a32fe47d9bc8a698607619b2e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:45:17.340740 30248 ts_tablet_manager.cc:1918] T a4fc0386d65744b4806f311357ac79ec P 8fe5cd5567524f4d928ba5b93ce819f6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:45:17.340823 30248 log.cc:1196] T a4fc0386d65744b4806f311357ac79ec P 8fe5cd5567524f4d928ba5b93ce819f6: Deleting WAL directory at /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/ts-2/wal/wals/a4fc0386d65744b4806f311357ac79ec
I20260806 22:45:17.341095 30248 ts_tablet_manager.cc:1939] T a4fc0386d65744b4806f311357ac79ec P 8fe5cd5567524f4d928ba5b93ce819f6: Deleting consensus metadata
I20260806 22:45:17.342097 30249 ts_tablet_manager.cc:1918] T a4fc0386d65744b4806f311357ac79ec P 0717372a32fe47d9bc8a698607619b2e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:45:17.342196 30249 log.cc:1196] T a4fc0386d65744b4806f311357ac79ec P 0717372a32fe47d9bc8a698607619b2e: Deleting WAL directory at /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/ts-1/wal/wals/a4fc0386d65744b4806f311357ac79ec
I20260806 22:45:17.342453 30249 ts_tablet_manager.cc:1939] T a4fc0386d65744b4806f311357ac79ec P 0717372a32fe47d9bc8a698607619b2e: Deleting consensus metadata
I20260806 22:45:17.343086 29440 catalog_manager.cc:4841] TS 0717372a32fe47d9bc8a698607619b2e (127.18.45.194:33361): tablet a4fc0386d65744b4806f311357ac79ec (table test-table [id=1a3ae0f06bcd4903a13c9b747ee4644c]) successfully deleted
I20260806 22:45:17.343545 29444 catalog_manager.cc:4841] TS 8fe5cd5567524f4d928ba5b93ce819f6 (127.18.45.195:37031): tablet a4fc0386d65744b4806f311357ac79ec (table test-table [id=1a3ae0f06bcd4903a13c9b747ee4644c]) successfully deleted
I20260806 22:45:17.343989 30250 ts_tablet_manager.cc:1905] T a4fc0386d65744b4806f311357ac79ec P 3746ae9981e84ed2ae955a063c7f9cde: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:45:17.346103 30250 ts_tablet_manager.cc:1918] T a4fc0386d65744b4806f311357ac79ec P 3746ae9981e84ed2ae955a063c7f9cde: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:45:17.346194 30250 log.cc:1196] T a4fc0386d65744b4806f311357ac79ec P 3746ae9981e84ed2ae955a063c7f9cde: Deleting WAL directory at /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/ts-0/wal/wals/a4fc0386d65744b4806f311357ac79ec
I20260806 22:45:17.346454 30250 ts_tablet_manager.cc:1939] T a4fc0386d65744b4806f311357ac79ec P 3746ae9981e84ed2ae955a063c7f9cde: Deleting consensus metadata
I20260806 22:45:17.347222 29440 catalog_manager.cc:4841] TS 3746ae9981e84ed2ae955a063c7f9cde (127.18.45.193:35205): tablet a4fc0386d65744b4806f311357ac79ec (table test-table [id=1a3ae0f06bcd4903a13c9b747ee4644c]) successfully deleted
Aug 06 22:45:17 dist-test-slave-h3ft krb5kdc[29399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056317, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 22:45:17 dist-test-slave-h3ft krb5kdc[29399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:45:17 dist-test-slave-h3ft krb5kdc[29399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
W20260806 22:45:17.372987 30271 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:45:17.365073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53068 (local address 127.18.45.254:41637)
0806 22:45:17.365167 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:17.365170 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:17.365277 (+ 107us) server_negotiation.cc:407] Connection header received
0806 22:45:17.365344 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:17.365345 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:17.365369 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:17.365460 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:17.366142 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:17.367081 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:17.368140 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:17.368480 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:17.368576 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:17.372888 (+ 4312us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53068: BlockingRecv error: recv got EOF from 127.0.0.1:53068 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 06 22:45:17 dist-test-slave-h3ft krb5kdc[29399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056317, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM
I20260806 22:45:17.384120 30271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:17.374442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53072 (local address 127.18.45.254:41637)
0806 22:45:17.374504 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:17.374507 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:17.374695 (+ 188us) server_negotiation.cc:407] Connection header received
0806 22:45:17.374757 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:17.374758 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:17.374772 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:17.374826 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:17.375676 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:17.376243 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:17.377262 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:17.377476 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:17.377548 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:17.380871 (+ 3323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:17.380873 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:17.380874 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:17.380884 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:17.382578 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:17.383154 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:17.383156 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:17.383156 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:17.383222 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:17.383469 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:17.383470 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:17.383470 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:17.383623 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:17.383711 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:17.383954 (+ 243us) server_negotiation.cc:299] Negotiation successful
0806 22:45:17.384036 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 22:45:17.386270 29466 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53072:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:45:17.386518 29466 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:45:17.388667 30079 tablet_service.cc:1467] Processing CreateTablet for tablet 302e1cdccc874858a25f7b90baa576f3 (DEFAULT_TABLE table=test-table [id=b037a4b14ac34dce82e36e95119c73f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:45:17.388676 29673 tablet_service.cc:1467] Processing CreateTablet for tablet 302e1cdccc874858a25f7b90baa576f3 (DEFAULT_TABLE table=test-table [id=b037a4b14ac34dce82e36e95119c73f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:45:17.388924 30079 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 302e1cdccc874858a25f7b90baa576f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:45:17.388922 29673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 302e1cdccc874858a25f7b90baa576f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:45:17.390442 30214 tablet_bootstrap.cc:492] T 302e1cdccc874858a25f7b90baa576f3 P 8fe5cd5567524f4d928ba5b93ce819f6: Bootstrap starting.
I20260806 22:45:17.390933 29873 tablet_service.cc:1467] Processing CreateTablet for tablet 302e1cdccc874858a25f7b90baa576f3 (DEFAULT_TABLE table=test-table [id=b037a4b14ac34dce82e36e95119c73f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:45:17.391114 30214 tablet_bootstrap.cc:654] T 302e1cdccc874858a25f7b90baa576f3 P 8fe5cd5567524f4d928ba5b93ce819f6: Neither blocks nor log segments found. Creating new log.
I20260806 22:45:17.391153 30215 tablet_bootstrap.cc:492] T 302e1cdccc874858a25f7b90baa576f3 P 3746ae9981e84ed2ae955a063c7f9cde: Bootstrap starting.
I20260806 22:45:17.391209 29873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 302e1cdccc874858a25f7b90baa576f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:45:17.391631 30215 tablet_bootstrap.cc:654] T 302e1cdccc874858a25f7b90baa576f3 P 3746ae9981e84ed2ae955a063c7f9cde: Neither blocks nor log segments found. Creating new log.
I20260806 22:45:17.392328 30214 tablet_bootstrap.cc:492] T 302e1cdccc874858a25f7b90baa576f3 P 8fe5cd5567524f4d928ba5b93ce819f6: No bootstrap required, opened a new log
I20260806 22:45:17.392329 30215 tablet_bootstrap.cc:492] T 302e1cdccc874858a25f7b90baa576f3 P 3746ae9981e84ed2ae955a063c7f9cde: No bootstrap required, opened a new log
I20260806 22:45:17.392388 30214 ts_tablet_manager.cc:1397] T 302e1cdccc874858a25f7b90baa576f3 P 8fe5cd5567524f4d928ba5b93ce819f6: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 22:45:17.392388 30215 ts_tablet_manager.cc:1397] T 302e1cdccc874858a25f7b90baa576f3 P 3746ae9981e84ed2ae955a063c7f9cde: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 22:45:17.392581 30215 raft_consensus.cc:348] T 302e1cdccc874858a25f7b90baa576f3 P 3746ae9981e84ed2ae955a063c7f9cde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3746ae9981e84ed2ae955a063c7f9cde" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 35205 } } peers { permanent_uuid: "0717372a32fe47d9bc8a698607619b2e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33361 } } peers { permanent_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 37031 } }
I20260806 22:45:17.392609 30214 raft_consensus.cc:348] T 302e1cdccc874858a25f7b90baa576f3 P 8fe5cd5567524f4d928ba5b93ce819f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3746ae9981e84ed2ae955a063c7f9cde" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 35205 } } peers { permanent_uuid: "0717372a32fe47d9bc8a698607619b2e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33361 } } peers { permanent_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 37031 } }
I20260806 22:45:17.392668 30215 raft_consensus.cc:374] T 302e1cdccc874858a25f7b90baa576f3 P 3746ae9981e84ed2ae955a063c7f9cde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:45:17.392689 30214 raft_consensus.cc:374] T 302e1cdccc874858a25f7b90baa576f3 P 8fe5cd5567524f4d928ba5b93ce819f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:45:17.392690 30215 raft_consensus.cc:729] T 302e1cdccc874858a25f7b90baa576f3 P 3746ae9981e84ed2ae955a063c7f9cde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3746ae9981e84ed2ae955a063c7f9cde, State: Initialized, Role: FOLLOWER
I20260806 22:45:17.392712 30214 raft_consensus.cc:729] T 302e1cdccc874858a25f7b90baa576f3 P 8fe5cd5567524f4d928ba5b93ce819f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fe5cd5567524f4d928ba5b93ce819f6, State: Initialized, Role: FOLLOWER
I20260806 22:45:17.392745 30215 consensus_queue.cc:260] T 302e1cdccc874858a25f7b90baa576f3 P 3746ae9981e84ed2ae955a063c7f9cde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3746ae9981e84ed2ae955a063c7f9cde" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 35205 } } peers { permanent_uuid: "0717372a32fe47d9bc8a698607619b2e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33361 } } peers { permanent_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 37031 } }
I20260806 22:45:17.392760 30214 consensus_queue.cc:260] T 302e1cdccc874858a25f7b90baa576f3 P 8fe5cd5567524f4d928ba5b93ce819f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3746ae9981e84ed2ae955a063c7f9cde" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 35205 } } peers { permanent_uuid: "0717372a32fe47d9bc8a698607619b2e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33361 } } peers { permanent_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 37031 } }
I20260806 22:45:17.392853 30214 ts_tablet_manager.cc:1428] T 302e1cdccc874858a25f7b90baa576f3 P 8fe5cd5567524f4d928ba5b93ce819f6: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:17.392853 30215 ts_tablet_manager.cc:1428] T 302e1cdccc874858a25f7b90baa576f3 P 3746ae9981e84ed2ae955a063c7f9cde: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260806 22:45:17.393354 30218 tablet_bootstrap.cc:492] T 302e1cdccc874858a25f7b90baa576f3 P 0717372a32fe47d9bc8a698607619b2e: Bootstrap starting.
I20260806 22:45:17.393939 30218 tablet_bootstrap.cc:654] T 302e1cdccc874858a25f7b90baa576f3 P 0717372a32fe47d9bc8a698607619b2e: Neither blocks nor log segments found. Creating new log.
I20260806 22:45:17.394851 30218 tablet_bootstrap.cc:492] T 302e1cdccc874858a25f7b90baa576f3 P 0717372a32fe47d9bc8a698607619b2e: No bootstrap required, opened a new log
I20260806 22:45:17.394920 30218 ts_tablet_manager.cc:1397] T 302e1cdccc874858a25f7b90baa576f3 P 0717372a32fe47d9bc8a698607619b2e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 22:45:17.395180 30218 raft_consensus.cc:348] T 302e1cdccc874858a25f7b90baa576f3 P 0717372a32fe47d9bc8a698607619b2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3746ae9981e84ed2ae955a063c7f9cde" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 35205 } } peers { permanent_uuid: "0717372a32fe47d9bc8a698607619b2e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33361 } } peers { permanent_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 37031 } }
I20260806 22:45:17.395288 30218 raft_consensus.cc:374] T 302e1cdccc874858a25f7b90baa576f3 P 0717372a32fe47d9bc8a698607619b2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:45:17.395316 30218 raft_consensus.cc:729] T 302e1cdccc874858a25f7b90baa576f3 P 0717372a32fe47d9bc8a698607619b2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0717372a32fe47d9bc8a698607619b2e, State: Initialized, Role: FOLLOWER
I20260806 22:45:17.395375 30218 consensus_queue.cc:260] T 302e1cdccc874858a25f7b90baa576f3 P 0717372a32fe47d9bc8a698607619b2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3746ae9981e84ed2ae955a063c7f9cde" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 35205 } } peers { permanent_uuid: "0717372a32fe47d9bc8a698607619b2e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33361 } } peers { permanent_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 37031 } }
I20260806 22:45:17.395532 30218 ts_tablet_manager.cc:1428] T 302e1cdccc874858a25f7b90baa576f3 P 0717372a32fe47d9bc8a698607619b2e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 22:45:17.414078 30222 raft_consensus.cc:482] T 302e1cdccc874858a25f7b90baa576f3 P 8fe5cd5567524f4d928ba5b93ce819f6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:45:17.414225 30222 raft_consensus.cc:504] T 302e1cdccc874858a25f7b90baa576f3 P 8fe5cd5567524f4d928ba5b93ce819f6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3746ae9981e84ed2ae955a063c7f9cde" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 35205 } } peers { permanent_uuid: "0717372a32fe47d9bc8a698607619b2e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33361 } } peers { permanent_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 37031 } }
I20260806 22:45:17.414732 30222 leader_election.cc:288] T 302e1cdccc874858a25f7b90baa576f3 P 8fe5cd5567524f4d928ba5b93ce819f6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3746ae9981e84ed2ae955a063c7f9cde (127.18.45.193:35205), 0717372a32fe47d9bc8a698607619b2e (127.18.45.194:33361)
I20260806 22:45:17.419760 30213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:17.415177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.195:38505 (local address 127.18.45.193:35205)
0806 22:45:17.415266 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:17.415270 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:17.415401 (+ 131us) server_negotiation.cc:407] Connection header received
0806 22:45:17.415634 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:17.415636 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:17.415649 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:17.415738 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:45:17.415999 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:17.416642 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:17.418520 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:17.419182 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:17.419273 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:17.419286 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:17.419595 (+ 309us) server_negotiation.cc:299] Negotiation successful
0806 22:45:17.419665 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 22:45:17.419795 30319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:17.415348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.194:33361 (local address 127.18.45.195:40841)
0806 22:45:17.415509 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:17.415521 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:17.415626 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:17.415843 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:17.415843 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:17.415846 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:45:17.416013 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:17.416015 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:17.417301 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:17.417302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:17.418496 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:17.418498 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:17.419434 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:17.419435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:17.419528 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:17.419534 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 22:45:17.419594 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 22:45:17.419670 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":24,"thread_start_us":79,"threads_started":1}
I20260806 22:45:17.419803 30317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:17.415097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.193:35205 (local address 127.18.45.195:38505)
0806 22:45:17.415310 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:17.415324 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:17.415523 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:17.415739 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:17.415740 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:17.415743 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:45:17.415876 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:17.415879 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:17.416751 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:17.416752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:17.418384 (+ 1632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:17.418387 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:17.419348 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:17.419350 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:17.419468 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:17.419474 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 22:45:17.419578 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 22:45:17.419653 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"mutex_wait_us":1,"thread_start_us":104,"threads_started":1}
I20260806 22:45:17.420011 30283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:17.415433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.195:40841 (local address 127.18.45.194:33361)
0806 22:45:17.415489 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:17.415492 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:17.415588 (+ 96us) server_negotiation.cc:407] Connection header received
0806 22:45:17.415715 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:17.415716 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:17.415728 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:17.415782 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:45:17.416121 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:17.416848 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:17.418588 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:17.419165 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:17.419243 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:17.419258 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:17.419871 (+ 613us) server_negotiation.cc:299] Negotiation successful
0806 22:45:17.419931 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 22:45:17.420354 29693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "302e1cdccc874858a25f7b90baa576f3" candidate_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3746ae9981e84ed2ae955a063c7f9cde" is_pre_election: true
I20260806 22:45:17.420420 29903 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "302e1cdccc874858a25f7b90baa576f3" candidate_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0717372a32fe47d9bc8a698607619b2e" is_pre_election: true
I20260806 22:45:17.420497 29693 raft_consensus.cc:2454] T 302e1cdccc874858a25f7b90baa576f3 P 3746ae9981e84ed2ae955a063c7f9cde [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fe5cd5567524f4d928ba5b93ce819f6 in term 0.
I20260806 22:45:17.420519 29903 raft_consensus.cc:2454] T 302e1cdccc874858a25f7b90baa576f3 P 0717372a32fe47d9bc8a698607619b2e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fe5cd5567524f4d928ba5b93ce819f6 in term 0.
I20260806 22:45:17.420852 30008 leader_election.cc:302] T 302e1cdccc874858a25f7b90baa576f3 P 8fe5cd5567524f4d928ba5b93ce819f6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0717372a32fe47d9bc8a698607619b2e, 8fe5cd5567524f4d928ba5b93ce819f6; no voters:
I20260806 22:45:17.421126 30222 raft_consensus.cc:2781] T 302e1cdccc874858a25f7b90baa576f3 P 8fe5cd5567524f4d928ba5b93ce819f6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:45:17.421185 30222 raft_consensus.cc:482] T 302e1cdccc874858a25f7b90baa576f3 P 8fe5cd5567524f4d928ba5b93ce819f6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:45:17.421213 30222 raft_consensus.cc:3037] T 302e1cdccc874858a25f7b90baa576f3 P 8fe5cd5567524f4d928ba5b93ce819f6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:45:17.422484 30222 raft_consensus.cc:504] T 302e1cdccc874858a25f7b90baa576f3 P 8fe5cd5567524f4d928ba5b93ce819f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3746ae9981e84ed2ae955a063c7f9cde" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 35205 } } peers { permanent_uuid: "0717372a32fe47d9bc8a698607619b2e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33361 } } peers { permanent_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 37031 } }
I20260806 22:45:17.422670 30222 leader_election.cc:288] T 302e1cdccc874858a25f7b90baa576f3 P 8fe5cd5567524f4d928ba5b93ce819f6 [CANDIDATE]: Term 1 election: Requested vote from peers 3746ae9981e84ed2ae955a063c7f9cde (127.18.45.193:35205), 0717372a32fe47d9bc8a698607619b2e (127.18.45.194:33361)
I20260806 22:45:17.422894 29693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "302e1cdccc874858a25f7b90baa576f3" candidate_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3746ae9981e84ed2ae955a063c7f9cde"
I20260806 22:45:17.423030 29693 raft_consensus.cc:3037] T 302e1cdccc874858a25f7b90baa576f3 P 3746ae9981e84ed2ae955a063c7f9cde [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:45:17.422901 29903 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "302e1cdccc874858a25f7b90baa576f3" candidate_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0717372a32fe47d9bc8a698607619b2e"
I20260806 22:45:17.423074 29903 raft_consensus.cc:3037] T 302e1cdccc874858a25f7b90baa576f3 P 0717372a32fe47d9bc8a698607619b2e [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:45:17.424117 29693 raft_consensus.cc:2454] T 302e1cdccc874858a25f7b90baa576f3 P 3746ae9981e84ed2ae955a063c7f9cde [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fe5cd5567524f4d928ba5b93ce819f6 in term 1.
I20260806 22:45:17.424117 29903 raft_consensus.cc:2454] T 302e1cdccc874858a25f7b90baa576f3 P 0717372a32fe47d9bc8a698607619b2e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fe5cd5567524f4d928ba5b93ce819f6 in term 1.
I20260806 22:45:17.424301 30008 leader_election.cc:302] T 302e1cdccc874858a25f7b90baa576f3 P 8fe5cd5567524f4d928ba5b93ce819f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0717372a32fe47d9bc8a698607619b2e, 8fe5cd5567524f4d928ba5b93ce819f6; no voters:
I20260806 22:45:17.424423 30222 raft_consensus.cc:2781] T 302e1cdccc874858a25f7b90baa576f3 P 8fe5cd5567524f4d928ba5b93ce819f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:45:17.424664 30222 raft_consensus.cc:686] T 302e1cdccc874858a25f7b90baa576f3 P 8fe5cd5567524f4d928ba5b93ce819f6 [term 1 LEADER]: Becoming Leader. State: Replica: 8fe5cd5567524f4d928ba5b93ce819f6, State: Running, Role: LEADER
I20260806 22:45:17.424768 30222 consensus_queue.cc:237] T 302e1cdccc874858a25f7b90baa576f3 P 8fe5cd5567524f4d928ba5b93ce819f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3746ae9981e84ed2ae955a063c7f9cde" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 35205 } } peers { permanent_uuid: "0717372a32fe47d9bc8a698607619b2e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33361 } } peers { permanent_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 37031 } }
I20260806 22:45:17.425709 29466 catalog_manager.cc:5495] T 302e1cdccc874858a25f7b90baa576f3 P 8fe5cd5567524f4d928ba5b93ce819f6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8fe5cd5567524f4d928ba5b93ce819f6 (127.18.45.195). New cstate: current_term: 1 leader_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3746ae9981e84ed2ae955a063c7f9cde" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 35205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0717372a32fe47d9bc8a698607619b2e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33361 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 37031 } health_report { overall_health: HEALTHY } } }
I20260806 22:45:17.432659 30210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:17.429092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36612 (local address 127.18.45.195:37031)
0806 22:45:17.429315 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:17.429317 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:17.429326 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:45:17.429370 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:17.429371 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:17.429398 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:17.429492 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:45:17.429743 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:17.430523 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:17.431587 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:17.431908 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:17.432013 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:17.432226 (+ 213us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:45:17.432397 (+ 171us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:45:17.432492 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:17.432513 (+ 21us) server_negotiation.cc:299] Negotiation successful
0806 22:45:17.432539 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189}
I20260806 22:45:17.434103 29693 raft_consensus.cc:1261] T 302e1cdccc874858a25f7b90baa576f3 P 3746ae9981e84ed2ae955a063c7f9cde [term 1 FOLLOWER]: Refusing update from remote peer 8fe5cd5567524f4d928ba5b93ce819f6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:45:17.434144 29903 raft_consensus.cc:1261] T 302e1cdccc874858a25f7b90baa576f3 P 0717372a32fe47d9bc8a698607619b2e [term 1 FOLLOWER]: Refusing update from remote peer 8fe5cd5567524f4d928ba5b93ce819f6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:45:17.434535 30321 consensus_queue.cc:1035] T 302e1cdccc874858a25f7b90baa576f3 P 8fe5cd5567524f4d928ba5b93ce819f6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3746ae9981e84ed2ae955a063c7f9cde" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 35205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:45:17.434718 30222 consensus_queue.cc:1035] T 302e1cdccc874858a25f7b90baa576f3 P 8fe5cd5567524f4d928ba5b93ce819f6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0717372a32fe47d9bc8a698607619b2e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:45:17.439745 29466 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53072:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:45:17.439893 29466 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53072:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:45:17.440724 29466 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f3e29cbdebbb43999c5d7cf6c549f7b8: Sending DeleteTablet for 3 replicas of tablet 302e1cdccc874858a25f7b90baa576f3
I20260806 22:45:17.441143 30079 tablet_service.cc:1514] Processing DeleteTablet for tablet 302e1cdccc874858a25f7b90baa576f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:45:17 UTC) from {username='oryx', principal='oryx/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:36598
I20260806 22:45:17.441257 29673 tablet_service.cc:1514] Processing DeleteTablet for tablet 302e1cdccc874858a25f7b90baa576f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:45:17 UTC) from {username='oryx', principal='oryx/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:54092
I20260806 22:45:17.441424 29873 tablet_service.cc:1514] Processing DeleteTablet for tablet 302e1cdccc874858a25f7b90baa576f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:45:17 UTC) from {username='oryx', principal='oryx/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:46334
I20260806 22:45:17.441567 30248 tablet_replica.cc:330] T 302e1cdccc874858a25f7b90baa576f3 P 8fe5cd5567524f4d928ba5b93ce819f6: stopping tablet replica
I20260806 22:45:17.441650 30248 raft_consensus.cc:2229] T 302e1cdccc874858a25f7b90baa576f3 P 8fe5cd5567524f4d928ba5b93ce819f6 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:45:17.441784 30248 raft_consensus.cc:2258] T 302e1cdccc874858a25f7b90baa576f3 P 8fe5cd5567524f4d928ba5b93ce819f6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:45:17.441907 30250 tablet_replica.cc:330] T 302e1cdccc874858a25f7b90baa576f3 P 3746ae9981e84ed2ae955a063c7f9cde: stopping tablet replica
I20260806 22:45:17.441993 30250 raft_consensus.cc:2229] T 302e1cdccc874858a25f7b90baa576f3 P 3746ae9981e84ed2ae955a063c7f9cde [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:45:17.442055 30250 raft_consensus.cc:2258] T 302e1cdccc874858a25f7b90baa576f3 P 3746ae9981e84ed2ae955a063c7f9cde [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:45:17.442154 30249 tablet_replica.cc:330] T 302e1cdccc874858a25f7b90baa576f3 P 0717372a32fe47d9bc8a698607619b2e: stopping tablet replica
I20260806 22:45:17.442230 30249 raft_consensus.cc:2229] T 302e1cdccc874858a25f7b90baa576f3 P 0717372a32fe47d9bc8a698607619b2e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:45:17.442297 30249 raft_consensus.cc:2258] T 302e1cdccc874858a25f7b90baa576f3 P 0717372a32fe47d9bc8a698607619b2e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:45:17.442626 30250 ts_tablet_manager.cc:1905] T 302e1cdccc874858a25f7b90baa576f3 P 3746ae9981e84ed2ae955a063c7f9cde: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:45:17.442915 29466 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53072:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:45:17.443157 29466 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:45:17.443763 30248 ts_tablet_manager.cc:1905] T 302e1cdccc874858a25f7b90baa576f3 P 8fe5cd5567524f4d928ba5b93ce819f6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:45:17.444769 30250 ts_tablet_manager.cc:1918] T 302e1cdccc874858a25f7b90baa576f3 P 3746ae9981e84ed2ae955a063c7f9cde: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:45:17.444839 30250 log.cc:1196] T 302e1cdccc874858a25f7b90baa576f3 P 3746ae9981e84ed2ae955a063c7f9cde: Deleting WAL directory at /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/ts-0/wal/wals/302e1cdccc874858a25f7b90baa576f3
I20260806 22:45:17.445115 30250 ts_tablet_manager.cc:1939] T 302e1cdccc874858a25f7b90baa576f3 P 3746ae9981e84ed2ae955a063c7f9cde: Deleting consensus metadata
I20260806 22:45:17.445736 30248 ts_tablet_manager.cc:1918] T 302e1cdccc874858a25f7b90baa576f3 P 8fe5cd5567524f4d928ba5b93ce819f6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:45:17.445808 30248 log.cc:1196] T 302e1cdccc874858a25f7b90baa576f3 P 8fe5cd5567524f4d928ba5b93ce819f6: Deleting WAL directory at /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/ts-2/wal/wals/302e1cdccc874858a25f7b90baa576f3
I20260806 22:45:17.446177 30248 ts_tablet_manager.cc:1939] T 302e1cdccc874858a25f7b90baa576f3 P 8fe5cd5567524f4d928ba5b93ce819f6: Deleting consensus metadata
I20260806 22:45:17.446605 29440 catalog_manager.cc:4841] TS 3746ae9981e84ed2ae955a063c7f9cde (127.18.45.193:35205): tablet 302e1cdccc874858a25f7b90baa576f3 (table test-table [id=b037a4b14ac34dce82e36e95119c73f9]) successfully deleted
I20260806 22:45:17.447079 30079 tablet_service.cc:1467] Processing CreateTablet for tablet fb3ab8e55b304a24bb4fc132fcd17a46 (DEFAULT_TABLE table=test-table [id=2c2ffa7a7fe448bb98835fa40bc89dac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:45:17.447120 29873 tablet_service.cc:1467] Processing CreateTablet for tablet fb3ab8e55b304a24bb4fc132fcd17a46 (DEFAULT_TABLE table=test-table [id=2c2ffa7a7fe448bb98835fa40bc89dac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:45:17.447217 30079 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb3ab8e55b304a24bb4fc132fcd17a46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:45:17.447288 29873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb3ab8e55b304a24bb4fc132fcd17a46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:45:17.448763 30214 tablet_bootstrap.cc:492] T fb3ab8e55b304a24bb4fc132fcd17a46 P 8fe5cd5567524f4d928ba5b93ce819f6: Bootstrap starting.
I20260806 22:45:17.449288 30214 tablet_bootstrap.cc:654] T fb3ab8e55b304a24bb4fc132fcd17a46 P 8fe5cd5567524f4d928ba5b93ce819f6: Neither blocks nor log segments found. Creating new log.
I20260806 22:45:17.449534 30218 tablet_bootstrap.cc:492] T fb3ab8e55b304a24bb4fc132fcd17a46 P 0717372a32fe47d9bc8a698607619b2e: Bootstrap starting.
I20260806 22:45:17.450112 30218 tablet_bootstrap.cc:654] T fb3ab8e55b304a24bb4fc132fcd17a46 P 0717372a32fe47d9bc8a698607619b2e: Neither blocks nor log segments found. Creating new log.
I20260806 22:45:17.450285 30214 tablet_bootstrap.cc:492] T fb3ab8e55b304a24bb4fc132fcd17a46 P 8fe5cd5567524f4d928ba5b93ce819f6: No bootstrap required, opened a new log
I20260806 22:45:17.450332 30214 ts_tablet_manager.cc:1397] T fb3ab8e55b304a24bb4fc132fcd17a46 P 8fe5cd5567524f4d928ba5b93ce819f6: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 22:45:17.450543 30214 raft_consensus.cc:348] T fb3ab8e55b304a24bb4fc132fcd17a46 P 8fe5cd5567524f4d928ba5b93ce819f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3746ae9981e84ed2ae955a063c7f9cde" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 35205 } } peers { permanent_uuid: "0717372a32fe47d9bc8a698607619b2e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33361 } } peers { permanent_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 37031 } }
I20260806 22:45:17.450628 30214 raft_consensus.cc:374] T fb3ab8e55b304a24bb4fc132fcd17a46 P 8fe5cd5567524f4d928ba5b93ce819f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:45:17.450650 30214 raft_consensus.cc:729] T fb3ab8e55b304a24bb4fc132fcd17a46 P 8fe5cd5567524f4d928ba5b93ce819f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fe5cd5567524f4d928ba5b93ce819f6, State: Initialized, Role: FOLLOWER
I20260806 22:45:17.450704 30214 consensus_queue.cc:260] T fb3ab8e55b304a24bb4fc132fcd17a46 P 8fe5cd5567524f4d928ba5b93ce819f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3746ae9981e84ed2ae955a063c7f9cde" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 35205 } } peers { permanent_uuid: "0717372a32fe47d9bc8a698607619b2e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33361 } } peers { permanent_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 37031 } }
I20260806 22:45:17.450703 29673 tablet_service.cc:1467] Processing CreateTablet for tablet fb3ab8e55b304a24bb4fc132fcd17a46 (DEFAULT_TABLE table=test-table [id=2c2ffa7a7fe448bb98835fa40bc89dac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:45:17.450807 30214 ts_tablet_manager.cc:1428] T fb3ab8e55b304a24bb4fc132fcd17a46 P 8fe5cd5567524f4d928ba5b93ce819f6: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260806 22:45:17.450874 29673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb3ab8e55b304a24bb4fc132fcd17a46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:45:17.452659 30215 tablet_bootstrap.cc:492] T fb3ab8e55b304a24bb4fc132fcd17a46 P 3746ae9981e84ed2ae955a063c7f9cde: Bootstrap starting.
I20260806 22:45:17.447021 29444 catalog_manager.cc:4841] TS 8fe5cd5567524f4d928ba5b93ce819f6 (127.18.45.195:37031): tablet 302e1cdccc874858a25f7b90baa576f3 (table test-table [id=b037a4b14ac34dce82e36e95119c73f9]) successfully deleted
I20260806 22:45:17.453064 30215 tablet_bootstrap.cc:654] T fb3ab8e55b304a24bb4fc132fcd17a46 P 3746ae9981e84ed2ae955a063c7f9cde: Neither blocks nor log segments found. Creating new log.
I20260806 22:45:17.454576 30249 ts_tablet_manager.cc:1905] T 302e1cdccc874858a25f7b90baa576f3 P 0717372a32fe47d9bc8a698607619b2e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:45:17.455789 30215 tablet_bootstrap.cc:492] T fb3ab8e55b304a24bb4fc132fcd17a46 P 3746ae9981e84ed2ae955a063c7f9cde: No bootstrap required, opened a new log
I20260806 22:45:17.455848 30218 tablet_bootstrap.cc:492] T fb3ab8e55b304a24bb4fc132fcd17a46 P 0717372a32fe47d9bc8a698607619b2e: No bootstrap required, opened a new log
I20260806 22:45:17.455868 30215 ts_tablet_manager.cc:1397] T fb3ab8e55b304a24bb4fc132fcd17a46 P 3746ae9981e84ed2ae955a063c7f9cde: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260806 22:45:17.455912 30218 ts_tablet_manager.cc:1397] T fb3ab8e55b304a24bb4fc132fcd17a46 P 0717372a32fe47d9bc8a698607619b2e: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20260806 22:45:17.456027 30215 raft_consensus.cc:348] T fb3ab8e55b304a24bb4fc132fcd17a46 P 3746ae9981e84ed2ae955a063c7f9cde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3746ae9981e84ed2ae955a063c7f9cde" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 35205 } } peers { permanent_uuid: "0717372a32fe47d9bc8a698607619b2e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33361 } } peers { permanent_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 37031 } }
I20260806 22:45:17.456122 30215 raft_consensus.cc:374] T fb3ab8e55b304a24bb4fc132fcd17a46 P 3746ae9981e84ed2ae955a063c7f9cde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:45:17.456143 30215 raft_consensus.cc:729] T fb3ab8e55b304a24bb4fc132fcd17a46 P 3746ae9981e84ed2ae955a063c7f9cde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3746ae9981e84ed2ae955a063c7f9cde, State: Initialized, Role: FOLLOWER
I20260806 22:45:17.456192 30215 consensus_queue.cc:260] T fb3ab8e55b304a24bb4fc132fcd17a46 P 3746ae9981e84ed2ae955a063c7f9cde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3746ae9981e84ed2ae955a063c7f9cde" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 35205 } } peers { permanent_uuid: "0717372a32fe47d9bc8a698607619b2e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33361 } } peers { permanent_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 37031 } }
I20260806 22:45:17.456148 30218 raft_consensus.cc:348] T fb3ab8e55b304a24bb4fc132fcd17a46 P 0717372a32fe47d9bc8a698607619b2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3746ae9981e84ed2ae955a063c7f9cde" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 35205 } } peers { permanent_uuid: "0717372a32fe47d9bc8a698607619b2e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33361 } } peers { permanent_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 37031 } }
I20260806 22:45:17.456240 30218 raft_consensus.cc:374] T fb3ab8e55b304a24bb4fc132fcd17a46 P 0717372a32fe47d9bc8a698607619b2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:45:17.456266 30218 raft_consensus.cc:729] T fb3ab8e55b304a24bb4fc132fcd17a46 P 0717372a32fe47d9bc8a698607619b2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0717372a32fe47d9bc8a698607619b2e, State: Initialized, Role: FOLLOWER
I20260806 22:45:17.456298 30215 ts_tablet_manager.cc:1428] T fb3ab8e55b304a24bb4fc132fcd17a46 P 3746ae9981e84ed2ae955a063c7f9cde: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:17.456326 30218 consensus_queue.cc:260] T fb3ab8e55b304a24bb4fc132fcd17a46 P 0717372a32fe47d9bc8a698607619b2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3746ae9981e84ed2ae955a063c7f9cde" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 35205 } } peers { permanent_uuid: "0717372a32fe47d9bc8a698607619b2e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33361 } } peers { permanent_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 37031 } }
I20260806 22:45:17.456440 30218 ts_tablet_manager.cc:1428] T fb3ab8e55b304a24bb4fc132fcd17a46 P 0717372a32fe47d9bc8a698607619b2e: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260806 22:45:17.458859 30249 ts_tablet_manager.cc:1918] T 302e1cdccc874858a25f7b90baa576f3 P 0717372a32fe47d9bc8a698607619b2e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:45:17.458946 30249 log.cc:1196] T 302e1cdccc874858a25f7b90baa576f3 P 0717372a32fe47d9bc8a698607619b2e: Deleting WAL directory at /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/ts-1/wal/wals/302e1cdccc874858a25f7b90baa576f3
I20260806 22:45:17.459264 30249 ts_tablet_manager.cc:1939] T 302e1cdccc874858a25f7b90baa576f3 P 0717372a32fe47d9bc8a698607619b2e: Deleting consensus metadata
I20260806 22:45:17.460053 29440 catalog_manager.cc:4841] TS 0717372a32fe47d9bc8a698607619b2e (127.18.45.194:33361): tablet 302e1cdccc874858a25f7b90baa576f3 (table test-table [id=b037a4b14ac34dce82e36e95119c73f9]) successfully deleted
I20260806 22:45:17.602345 30223 raft_consensus.cc:482] T fb3ab8e55b304a24bb4fc132fcd17a46 P 3746ae9981e84ed2ae955a063c7f9cde [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:45:17.602542 30223 raft_consensus.cc:504] T fb3ab8e55b304a24bb4fc132fcd17a46 P 3746ae9981e84ed2ae955a063c7f9cde [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3746ae9981e84ed2ae955a063c7f9cde" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 35205 } } peers { permanent_uuid: "0717372a32fe47d9bc8a698607619b2e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33361 } } peers { permanent_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 37031 } }
I20260806 22:45:17.602793 30223 leader_election.cc:288] T fb3ab8e55b304a24bb4fc132fcd17a46 P 3746ae9981e84ed2ae955a063c7f9cde [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0717372a32fe47d9bc8a698607619b2e (127.18.45.194:33361), 8fe5cd5567524f4d928ba5b93ce819f6 (127.18.45.195:37031)
I20260806 22:45:17.603204 29903 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb3ab8e55b304a24bb4fc132fcd17a46" candidate_uuid: "3746ae9981e84ed2ae955a063c7f9cde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0717372a32fe47d9bc8a698607619b2e" is_pre_election: true
I20260806 22:45:17.603229 30104 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb3ab8e55b304a24bb4fc132fcd17a46" candidate_uuid: "3746ae9981e84ed2ae955a063c7f9cde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" is_pre_election: true
I20260806 22:45:17.603356 29903 raft_consensus.cc:2454] T fb3ab8e55b304a24bb4fc132fcd17a46 P 0717372a32fe47d9bc8a698607619b2e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3746ae9981e84ed2ae955a063c7f9cde in term 0.
I20260806 22:45:17.603355 30104 raft_consensus.cc:2454] T fb3ab8e55b304a24bb4fc132fcd17a46 P 8fe5cd5567524f4d928ba5b93ce819f6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3746ae9981e84ed2ae955a063c7f9cde in term 0.
I20260806 22:45:17.603552 29602 leader_election.cc:302] T fb3ab8e55b304a24bb4fc132fcd17a46 P 3746ae9981e84ed2ae955a063c7f9cde [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3746ae9981e84ed2ae955a063c7f9cde, 8fe5cd5567524f4d928ba5b93ce819f6; no voters:
I20260806 22:45:17.603703 30223 raft_consensus.cc:2781] T fb3ab8e55b304a24bb4fc132fcd17a46 P 3746ae9981e84ed2ae955a063c7f9cde [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:45:17.603750 30223 raft_consensus.cc:482] T fb3ab8e55b304a24bb4fc132fcd17a46 P 3746ae9981e84ed2ae955a063c7f9cde [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:45:17.603783 30223 raft_consensus.cc:3037] T fb3ab8e55b304a24bb4fc132fcd17a46 P 3746ae9981e84ed2ae955a063c7f9cde [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:45:17.605085 30223 raft_consensus.cc:504] T fb3ab8e55b304a24bb4fc132fcd17a46 P 3746ae9981e84ed2ae955a063c7f9cde [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3746ae9981e84ed2ae955a063c7f9cde" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 35205 } } peers { permanent_uuid: "0717372a32fe47d9bc8a698607619b2e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33361 } } peers { permanent_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 37031 } }
I20260806 22:45:17.605289 30223 leader_election.cc:288] T fb3ab8e55b304a24bb4fc132fcd17a46 P 3746ae9981e84ed2ae955a063c7f9cde [CANDIDATE]: Term 1 election: Requested vote from peers 0717372a32fe47d9bc8a698607619b2e (127.18.45.194:33361), 8fe5cd5567524f4d928ba5b93ce819f6 (127.18.45.195:37031)
I20260806 22:45:17.605484 29903 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb3ab8e55b304a24bb4fc132fcd17a46" candidate_uuid: "3746ae9981e84ed2ae955a063c7f9cde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0717372a32fe47d9bc8a698607619b2e"
I20260806 22:45:17.605623 29903 raft_consensus.cc:3037] T fb3ab8e55b304a24bb4fc132fcd17a46 P 0717372a32fe47d9bc8a698607619b2e [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:45:17.605581 30104 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb3ab8e55b304a24bb4fc132fcd17a46" candidate_uuid: "3746ae9981e84ed2ae955a063c7f9cde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fe5cd5567524f4d928ba5b93ce819f6"
I20260806 22:45:17.605698 30104 raft_consensus.cc:3037] T fb3ab8e55b304a24bb4fc132fcd17a46 P 8fe5cd5567524f4d928ba5b93ce819f6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:45:17.606629 30104 raft_consensus.cc:2454] T fb3ab8e55b304a24bb4fc132fcd17a46 P 8fe5cd5567524f4d928ba5b93ce819f6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3746ae9981e84ed2ae955a063c7f9cde in term 1.
I20260806 22:45:17.606865 29602 leader_election.cc:302] T fb3ab8e55b304a24bb4fc132fcd17a46 P 3746ae9981e84ed2ae955a063c7f9cde [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3746ae9981e84ed2ae955a063c7f9cde, 8fe5cd5567524f4d928ba5b93ce819f6; no voters:
I20260806 22:45:17.606930 29903 raft_consensus.cc:2454] T fb3ab8e55b304a24bb4fc132fcd17a46 P 0717372a32fe47d9bc8a698607619b2e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3746ae9981e84ed2ae955a063c7f9cde in term 1.
I20260806 22:45:17.607043 30223 raft_consensus.cc:2781] T fb3ab8e55b304a24bb4fc132fcd17a46 P 3746ae9981e84ed2ae955a063c7f9cde [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:45:17.607206 30223 raft_consensus.cc:686] T fb3ab8e55b304a24bb4fc132fcd17a46 P 3746ae9981e84ed2ae955a063c7f9cde [term 1 LEADER]: Becoming Leader. State: Replica: 3746ae9981e84ed2ae955a063c7f9cde, State: Running, Role: LEADER
I20260806 22:45:17.607287 30223 consensus_queue.cc:237] T fb3ab8e55b304a24bb4fc132fcd17a46 P 3746ae9981e84ed2ae955a063c7f9cde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3746ae9981e84ed2ae955a063c7f9cde" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 35205 } } peers { permanent_uuid: "0717372a32fe47d9bc8a698607619b2e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33361 } } peers { permanent_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 37031 } }
I20260806 22:45:17.608239 29465 catalog_manager.cc:5495] T fb3ab8e55b304a24bb4fc132fcd17a46 P 3746ae9981e84ed2ae955a063c7f9cde reported cstate change: term changed from 0 to 1, leader changed from <none> to 3746ae9981e84ed2ae955a063c7f9cde (127.18.45.193). New cstate: current_term: 1 leader_uuid: "3746ae9981e84ed2ae955a063c7f9cde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3746ae9981e84ed2ae955a063c7f9cde" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 35205 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0717372a32fe47d9bc8a698607619b2e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33361 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 37031 } health_report { overall_health: UNKNOWN } } }
I20260806 22:45:17.654003 30213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:17.650387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54118 (local address 127.18.45.193:35205)
0806 22:45:17.650467 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:17.650469 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:17.650479 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:45:17.650515 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:17.650516 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:17.650550 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:17.650606 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:45:17.650910 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:17.651993 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:17.653036 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:17.653326 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:17.653409 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:17.653669 (+ 260us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:45:17.653714 (+ 45us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:45:17.653763 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:17.653873 (+ 110us) server_negotiation.cc:299] Negotiation successful
0806 22:45:17.653902 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 22:45:17.660514 30337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:17.657537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.194:33361 (local address 127.18.45.193:33541)
0806 22:45:17.657759 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:17.657772 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:17.657851 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:17.658004 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:17.658005 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:17.658007 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:45:17.658078 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:17.658079 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:17.658727 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:17.658727 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:17.659546 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:17.659549 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:17.659986 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:17.659987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:17.660046 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:17.660066 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:45:17.660293 (+ 227us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:45:17.660297 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:45:17.660354 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 22:45:17.660433 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":136,"threads_started":1}
I20260806 22:45:17.660501 30283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:17.657659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:33541 (local address 127.18.45.194:33361)
0806 22:45:17.657751 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:17.657753 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:17.657833 (+ 80us) server_negotiation.cc:407] Connection header received
0806 22:45:17.657913 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:17.657914 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:17.657928 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:17.657988 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:45:17.658139 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:17.658657 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:17.659646 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:17.659888 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:17.659954 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:17.660161 (+ 207us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:45:17.660199 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:45:17.660248 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:17.660359 (+ 111us) server_negotiation.cc:299] Negotiation successful
0806 22:45:17.660384 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 22:45:17.663767 30104 raft_consensus.cc:1261] T fb3ab8e55b304a24bb4fc132fcd17a46 P 8fe5cd5567524f4d928ba5b93ce819f6 [term 1 FOLLOWER]: Refusing update from remote peer 3746ae9981e84ed2ae955a063c7f9cde: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:45:17.663813 29903 raft_consensus.cc:1261] T fb3ab8e55b304a24bb4fc132fcd17a46 P 0717372a32fe47d9bc8a698607619b2e [term 1 FOLLOWER]: Refusing update from remote peer 3746ae9981e84ed2ae955a063c7f9cde: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:45:17.664069 30331 consensus_queue.cc:1035] T fb3ab8e55b304a24bb4fc132fcd17a46 P 3746ae9981e84ed2ae955a063c7f9cde [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fe5cd5567524f4d928ba5b93ce819f6" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 37031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:45:17.664232 30223 consensus_queue.cc:1035] T fb3ab8e55b304a24bb4fc132fcd17a46 P 3746ae9981e84ed2ae955a063c7f9cde [LEADER]: Connected to new peer: Peer: permanent_uuid: "0717372a32fe47d9bc8a698607619b2e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:45:17.672834 30293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:17.669990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.193:35205 (local address 127.18.45.194:59879)
0806 22:45:17.670064 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:17.670079 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:17.670126 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:17.670265 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:17.670266 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:17.670268 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 22:45:17.670332 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:17.670334 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:17.671062 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:17.671063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:17.671828 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:17.671831 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:17.672304 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:17.672305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:17.672376 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:17.672387 (+ 11us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 22:45:17.672675 (+ 288us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 22:45:17.672677 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:45:17.672714 (+ 37us) client_negotiation.cc:240] Negotiation successful
0806 22:45:17.672771 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260806 22:45:17.672828 30213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:17.670063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:59879 (local address 127.18.45.193:35205)
0806 22:45:17.670135 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:17.670138 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:17.670149 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:45:17.670178 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:17.670178 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:17.670192 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:17.670263 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:45:17.670415 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:17.670933 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:17.671907 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:17.672218 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:17.672293 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:17.672546 (+ 253us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:45:17.672608 (+ 62us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:45:17.672673 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:17.672718 (+ 45us) server_negotiation.cc:299] Negotiation successful
0806 22:45:17.672744 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 22:45:17.679436 29465 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53072:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:45:17.679581 29465 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53072:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:45:17.680371 29465 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f3e29cbdebbb43999c5d7cf6c549f7b8: Sending DeleteTablet for 3 replicas of tablet fb3ab8e55b304a24bb4fc132fcd17a46
I20260806 22:45:17.680658 29873 tablet_service.cc:1514] Processing DeleteTablet for tablet fb3ab8e55b304a24bb4fc132fcd17a46 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:45:17 UTC) from {username='oryx', principal='oryx/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:46334
I20260806 22:45:17.680699 30079 tablet_service.cc:1514] Processing DeleteTablet for tablet fb3ab8e55b304a24bb4fc132fcd17a46 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:45:17 UTC) from {username='oryx', principal='oryx/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:36598
I20260806 22:45:17.680774 30249 tablet_replica.cc:330] T fb3ab8e55b304a24bb4fc132fcd17a46 P 0717372a32fe47d9bc8a698607619b2e: stopping tablet replica
I20260806 22:45:17.680763 29673 tablet_service.cc:1514] Processing DeleteTablet for tablet fb3ab8e55b304a24bb4fc132fcd17a46 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:45:17 UTC) from {username='oryx', principal='oryx/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:54092
I20260806 22:45:17.680835 30249 raft_consensus.cc:2229] T fb3ab8e55b304a24bb4fc132fcd17a46 P 0717372a32fe47d9bc8a698607619b2e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:45:17.680888 30249 raft_consensus.cc:2258] T fb3ab8e55b304a24bb4fc132fcd17a46 P 0717372a32fe47d9bc8a698607619b2e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:45:17.680884 30248 tablet_replica.cc:330] T fb3ab8e55b304a24bb4fc132fcd17a46 P 8fe5cd5567524f4d928ba5b93ce819f6: stopping tablet replica
I20260806 22:45:17.680907 30250 tablet_replica.cc:330] T fb3ab8e55b304a24bb4fc132fcd17a46 P 3746ae9981e84ed2ae955a063c7f9cde: stopping tablet replica
I20260806 22:45:17.680984 30248 raft_consensus.cc:2229] T fb3ab8e55b304a24bb4fc132fcd17a46 P 8fe5cd5567524f4d928ba5b93ce819f6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:45:17.680984 30250 raft_consensus.cc:2229] T fb3ab8e55b304a24bb4fc132fcd17a46 P 3746ae9981e84ed2ae955a063c7f9cde [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:45:17.681030 30248 raft_consensus.cc:2258] T fb3ab8e55b304a24bb4fc132fcd17a46 P 8fe5cd5567524f4d928ba5b93ce819f6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:45:17.681051 30250 raft_consensus.cc:2258] T fb3ab8e55b304a24bb4fc132fcd17a46 P 3746ae9981e84ed2ae955a063c7f9cde [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:45:17.681452 30250 ts_tablet_manager.cc:1905] T fb3ab8e55b304a24bb4fc132fcd17a46 P 3746ae9981e84ed2ae955a063c7f9cde: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:45:17.681574 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 29563
I20260806 22:45:17.681653 30249 ts_tablet_manager.cc:1905] T fb3ab8e55b304a24bb4fc132fcd17a46 P 0717372a32fe47d9bc8a698607619b2e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:45:17.691493 30248 ts_tablet_manager.cc:1905] T fb3ab8e55b304a24bb4fc132fcd17a46 P 8fe5cd5567524f4d928ba5b93ce819f6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:45:17.693184 30248 ts_tablet_manager.cc:1918] T fb3ab8e55b304a24bb4fc132fcd17a46 P 8fe5cd5567524f4d928ba5b93ce819f6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:45:17.693259 30248 log.cc:1196] T fb3ab8e55b304a24bb4fc132fcd17a46 P 8fe5cd5567524f4d928ba5b93ce819f6: Deleting WAL directory at /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/ts-2/wal/wals/fb3ab8e55b304a24bb4fc132fcd17a46
I20260806 22:45:17.693540 30248 ts_tablet_manager.cc:1939] T fb3ab8e55b304a24bb4fc132fcd17a46 P 8fe5cd5567524f4d928ba5b93ce819f6: Deleting consensus metadata
I20260806 22:45:17.694078 29444 catalog_manager.cc:4841] TS 8fe5cd5567524f4d928ba5b93ce819f6 (127.18.45.195:37031): tablet fb3ab8e55b304a24bb4fc132fcd17a46 (table test-table [id=2c2ffa7a7fe448bb98835fa40bc89dac]) successfully deleted
I20260806 22:45:17.695230 30249 ts_tablet_manager.cc:1918] T fb3ab8e55b304a24bb4fc132fcd17a46 P 0717372a32fe47d9bc8a698607619b2e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 22:45:17.695304 30249 log.cc:1196] T fb3ab8e55b304a24bb4fc132fcd17a46 P 0717372a32fe47d9bc8a698607619b2e: Deleting WAL directory at /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786056275803658-18615-0/minicluster-data/ts-1/wal/wals/fb3ab8e55b304a24bb4fc132fcd17a46
I20260806 22:45:17.695537 30249 ts_tablet_manager.cc:1939] T fb3ab8e55b304a24bb4fc132fcd17a46 P 0717372a32fe47d9bc8a698607619b2e: Deleting consensus metadata
I20260806 22:45:17.696049 29440 catalog_manager.cc:4841] TS 0717372a32fe47d9bc8a698607619b2e (127.18.45.194:33361): tablet fb3ab8e55b304a24bb4fc132fcd17a46 (table test-table [id=2c2ffa7a7fe448bb98835fa40bc89dac]) successfully deleted
W20260806 22:45:17.703908 29440 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.45.193:35205 (error 108)
I20260806 22:45:17.704344 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 29776
I20260806 22:45:17.704952 30346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:17.704622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.193:35205 (local address 127.0.0.1:54124)
0806 22:45:17.704831 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:17.704888 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.193:35205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":77,"threads_started":1}
W20260806 22:45:17.705263 29440 catalog_manager.cc:4584] TS 3746ae9981e84ed2ae955a063c7f9cde (127.18.45.193:35205): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fb3ab8e55b304a24bb4fc132fcd17a46: Network error: Client connection negotiation failed: client connection to 127.18.45.193:35205: connect: Connection refused (error 111)
I20260806 22:45:17.727429 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 29986
I20260806 22:45:17.750020 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 29419
2026-08-06T22:45:17Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3546 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056275803658-18615-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:45:17 dist-test-slave-h3ft krb5kdc[30350](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:45:17 dist-test-slave-h3ft krb5kdc[30350](info): set up 2 sockets
Aug 06 22:45:17 dist-test-slave-h3ft krb5kdc[30350](info): commencing operation
krb5kdc: starting...
W20260806 22:45:19.057111 18615 mini_kdc.cc:121] Time spent starting KDC: real 1.273s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Aug 06 22:45:19 dist-test-slave-h3ft krb5kdc[30350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056319, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:45:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:45:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.45.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab.
Entry for principal kudu/127.18.45.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab.
I20260806 22:45:19.210894 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056275803658-18615-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056275803658-18615-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056275803658-18615-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:35955
--webserver_interface=127.18.45.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:40383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.45.254:35955
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:45:19.308725 30369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:45:19.309049 30369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:45:19.309088 30369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:45:19.310341 30369 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:45:19.310413 30369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:45:19.310437 30369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:45:19.310459 30369 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:45:19.310489 30369 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:45:19.312160 30369 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:40383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056275803658-18615-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056275803658-18615-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.45.254:35955
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:35955
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.45.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30369
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786056275803658-18615-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:45:19.312593 30369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:45:19.312906 30369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:45:19.318681 30380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:19.318681 30377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:19.318794 30376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:19.319139 30369 server_base.cc:1034] running on GCE node
I20260806 22:45:19.319458 30369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:45:19.319842 30369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:45:19.321028 30369 hybrid_clock.cc:648] HybridClock initialized: now 1786056319320996 us; error 49 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-06T22:45:19Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1579 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:45:19 dist-test-slave-h3ft krb5kdc[30386](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:45:19 dist-test-slave-h3ft krb5kdc[30386](info): set up 2 sockets
Aug 06 22:45:19 dist-test-slave-h3ft krb5kdc[30386](info): commencing operation
krb5kdc: starting...
W20260806 22:45:20.655472 18615 mini_kdc.cc:121] Time spent starting KDC: real 1.292s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Aug 06 22:45:20 dist-test-slave-h3ft krb5kdc[30386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056320, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:45:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:45:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.45.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab.
Entry for principal kudu/127.18.45.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab.
I20260806 22:45:20.824034 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:40873
--webserver_interface=127.18.45.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:41485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.45.254:40873
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:45:20.923974 30407 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:45:20.924226 30407 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:45:20.924252 30407 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:45:20.925485 30407 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:45:20.925535 30407 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:45:20.925550 30407 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:45:20.925563 30407 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:45:20.925576 30407 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:45:20.927240 30407 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:41485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.45.254:40873
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:40873
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.45.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30407
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:45:20.927556 30407 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:45:20.927840 30407 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:45:20.933676 30415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:20.933669 30414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:20.933688 30418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:20.934129 30407 server_base.cc:1034] running on GCE node
I20260806 22:45:20.934474 30407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:45:20.934823 30407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:45:20.936053 30407 hybrid_clock.cc:648] HybridClock initialized: now 1786056320936038 us; error 64 us; skew 500 ppm
Aug 06 22:45:20 dist-test-slave-h3ft krb5kdc[30386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056320, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:45:20.939296 30407 init.cc:377] Logged in from keytab as kudu/127.18.45.254@KRBTEST.COM (short username kudu)
I20260806 22:45:20.939875 30407 webserver.cc:460] Webserver started at http://127.18.45.254:44781/ using document root <none> and password file <none>
I20260806 22:45:20.940171 30407 fs_manager.cc:359] Metadata directory not provided
I20260806 22:45:20.940279 30407 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:45:20.940448 30407 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:45:20.941730 30407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/master-0/data/instance:
uuid: "652683bd3ead4e4b896e35b45fc31dc8"
format_stamp: "Formatted at 2026-08-06 22:45:20 on dist-test-slave-h3ft"
server_key: "92b471661d08f3ead691860047c4e7dc"
server_key_iv: "ca5ab7c9d9f28e6d287485a5d2ad3b81"
server_key_version: "encryptionkey@0"
I20260806 22:45:20.942153 30407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/master-0/wal/instance:
uuid: "652683bd3ead4e4b896e35b45fc31dc8"
format_stamp: "Formatted at 2026-08-06 22:45:20 on dist-test-slave-h3ft"
server_key: "92b471661d08f3ead691860047c4e7dc"
server_key_iv: "ca5ab7c9d9f28e6d287485a5d2ad3b81"
server_key_version: "encryptionkey@0"
I20260806 22:45:20.943832 30407 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260806 22:45:20.944798 30429 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:20.945061 30407 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:45:20.945184 30407 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/master-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/master-0/wal
uuid: "652683bd3ead4e4b896e35b45fc31dc8"
format_stamp: "Formatted at 2026-08-06 22:45:20 on dist-test-slave-h3ft"
server_key: "92b471661d08f3ead691860047c4e7dc"
server_key_iv: "ca5ab7c9d9f28e6d287485a5d2ad3b81"
server_key_version: "encryptionkey@0"
I20260806 22:45:20.945267 30407 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:45:20.960578 30407 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:45:20.963277 30407 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:45:20.963487 30407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:45:20.968796 30407 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.254:40873
I20260806 22:45:20.968843 30501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.254:40873 every 8 connection(s)
I20260806 22:45:20.969326 30407 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
I20260806 22:45:20.970551 30502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:45:20.971944 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 30407
I20260806 22:45:20.972111 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/master-0/wal/instance
I20260806 22:45:20.972370 18615 external_mini_cluster.cc:1428] Setting key b89e5b4c3722d9c0fcbbac2a6deecdf6
I20260806 22:45:20.974370 30502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 652683bd3ead4e4b896e35b45fc31dc8: Bootstrap starting.
I20260806 22:45:20.975440 30502 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 652683bd3ead4e4b896e35b45fc31dc8: Neither blocks nor log segments found. Creating new log.
I20260806 22:45:20.975823 30502 log.cc:824] T 00000000000000000000000000000000 P 652683bd3ead4e4b896e35b45fc31dc8: Log is configured to *not* fsync() on all Append() calls
I20260806 22:45:20.976759 30502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 652683bd3ead4e4b896e35b45fc31dc8: No bootstrap required, opened a new log
I20260806 22:45:20.979897 30502 raft_consensus.cc:348] T 00000000000000000000000000000000 P 652683bd3ead4e4b896e35b45fc31dc8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "652683bd3ead4e4b896e35b45fc31dc8" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 40873 } }
I20260806 22:45:20.980130 30502 raft_consensus.cc:374] T 00000000000000000000000000000000 P 652683bd3ead4e4b896e35b45fc31dc8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:45:20.980175 30502 raft_consensus.cc:729] T 00000000000000000000000000000000 P 652683bd3ead4e4b896e35b45fc31dc8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652683bd3ead4e4b896e35b45fc31dc8, State: Initialized, Role: FOLLOWER
I20260806 22:45:20.980361 30502 consensus_queue.cc:260] T 00000000000000000000000000000000 P 652683bd3ead4e4b896e35b45fc31dc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "652683bd3ead4e4b896e35b45fc31dc8" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 40873 } }
I20260806 22:45:20.980489 30502 raft_consensus.cc:388] T 00000000000000000000000000000000 P 652683bd3ead4e4b896e35b45fc31dc8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:45:20.980563 30502 raft_consensus.cc:482] T 00000000000000000000000000000000 P 652683bd3ead4e4b896e35b45fc31dc8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:45:20.980624 30502 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 652683bd3ead4e4b896e35b45fc31dc8 [term 0 FOLLOWER]: Advancing to term 1
Aug 06 22:45:20 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056320, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:45:20.981437 30502 raft_consensus.cc:504] T 00000000000000000000000000000000 P 652683bd3ead4e4b896e35b45fc31dc8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "652683bd3ead4e4b896e35b45fc31dc8" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 40873 } }
I20260806 22:45:20.981600 30502 leader_election.cc:302] T 00000000000000000000000000000000 P 652683bd3ead4e4b896e35b45fc31dc8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 652683bd3ead4e4b896e35b45fc31dc8; no voters:
I20260806 22:45:20.981843 30502 leader_election.cc:288] T 00000000000000000000000000000000 P 652683bd3ead4e4b896e35b45fc31dc8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:45:20.981921 30508 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 652683bd3ead4e4b896e35b45fc31dc8 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:45:20.982139 30508 raft_consensus.cc:686] T 00000000000000000000000000000000 P 652683bd3ead4e4b896e35b45fc31dc8 [term 1 LEADER]: Becoming Leader. State: Replica: 652683bd3ead4e4b896e35b45fc31dc8, State: Running, Role: LEADER
I20260806 22:45:20.982245 30502 sys_catalog.cc:564] T 00000000000000000000000000000000 P 652683bd3ead4e4b896e35b45fc31dc8 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:45:20.982302 30508 consensus_queue.cc:237] T 00000000000000000000000000000000 P 652683bd3ead4e4b896e35b45fc31dc8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "652683bd3ead4e4b896e35b45fc31dc8" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 40873 } }
I20260806 22:45:20.982906 30509 sys_catalog.cc:455] T 00000000000000000000000000000000 P 652683bd3ead4e4b896e35b45fc31dc8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "652683bd3ead4e4b896e35b45fc31dc8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "652683bd3ead4e4b896e35b45fc31dc8" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 40873 } } }
I20260806 22:45:20.983050 30509 sys_catalog.cc:458] T 00000000000000000000000000000000 P 652683bd3ead4e4b896e35b45fc31dc8 [sys.catalog]: This master's current role is: LEADER
I20260806 22:45:20.983332 30519 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:45:20.984066 30519 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:45:20.984165 30511 sys_catalog.cc:455] T 00000000000000000000000000000000 P 652683bd3ead4e4b896e35b45fc31dc8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 652683bd3ead4e4b896e35b45fc31dc8. Latest consensus state: current_term: 1 leader_uuid: "652683bd3ead4e4b896e35b45fc31dc8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "652683bd3ead4e4b896e35b45fc31dc8" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 40873 } } }
I20260806 22:45:20.984366 30511 sys_catalog.cc:458] T 00000000000000000000000000000000 P 652683bd3ead4e4b896e35b45fc31dc8 [sys.catalog]: This master's current role is: LEADER
I20260806 22:45:20.986069 30519 catalog_manager.cc:1347] Generated new cluster ID: 6116af451d3e42fcbb37cbe429c2f7f1
I20260806 22:45:20.986129 30519 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:45:20.986917 30505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:20.973138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54260 (local address 127.18.45.254:40873)
0806 22:45:20.973334 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:20.973344 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:20.973375 (+ 31us) server_negotiation.cc:407] Connection header received
0806 22:45:20.974078 (+ 703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:20.974086 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:20.974145 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:20.974287 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:20.975352 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:20.976449 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:20.977679 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:20.978154 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:20.978245 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:20.981710 (+ 3465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:20.981716 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:20.981717 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:20.981734 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:20.984695 (+ 2961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:20.985191 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:20.985196 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:20.985196 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:20.985332 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:20.985614 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:20.985616 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:20.985617 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:20.985981 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:20.986159 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:20.986493 (+ 334us) server_negotiation.cc:299] Negotiation successful
0806 22:45:20.986672 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":97,"threads_started":1}
I20260806 22:45:21.006249 30519 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:45:21.006897 30519 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:45:21.012063 30519 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 652683bd3ead4e4b896e35b45fc31dc8: Generated new TSK 0
I20260806 22:45:21.012296 30519 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.45.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab.
Entry for principal kudu/127.18.45.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab.
I20260806 22:45:21.047748 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.193:0
--local_ip_for_outbound_sockets=127.18.45.193
--webserver_interface=127.18.45.193
--webserver_port=0
--tserver_master_addrs=127.18.45.254:40873
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab
--principal=kudu/127.18.45.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:41485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:45:21.150180 30538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:45:21.150462 30538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:45:21.150493 30538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:45:21.151906 30538 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:45:21.151957 30538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:45:21.151988 30538 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.193
I20260806 22:45:21.153589 30538 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:41485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab
--principal=kudu/127.18.45.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.45.193
--webserver_port=0
--tserver_master_addrs=127.18.45.254:40873
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30538
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.193
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:45:21.153900 30538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:45:21.154212 30538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:45:21.160948 30547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:21.160965 30545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:21.160966 30544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:21.161144 30538 server_base.cc:1034] running on GCE node
I20260806 22:45:21.161530 30538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:45:21.161785 30538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:45:21.162931 30538 hybrid_clock.cc:648] HybridClock initialized: now 1786056321162912 us; error 43 us; skew 500 ppm
Aug 06 22:45:21 dist-test-slave-h3ft krb5kdc[30386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056321, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:45:21.166051 30538 init.cc:377] Logged in from keytab as kudu/127.18.45.193@KRBTEST.COM (short username kudu)
I20260806 22:45:21.166579 30538 webserver.cc:460] Webserver started at http://127.18.45.193:36015/ using document root <none> and password file <none>
I20260806 22:45:21.166844 30538 fs_manager.cc:359] Metadata directory not provided
I20260806 22:45:21.166919 30538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:45:21.167119 30538 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:45:21.168381 30538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-0/data/instance:
uuid: "01763e39229d45b582248a73dfe8a9c4"
format_stamp: "Formatted at 2026-08-06 22:45:21 on dist-test-slave-h3ft"
server_key: "5b8529946cf4fff58ca3c4fadb383e34"
server_key_iv: "0b58d2833cf414aa4ba978950de0123e"
server_key_version: "encryptionkey@0"
I20260806 22:45:21.168808 30538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-0/wal/instance:
uuid: "01763e39229d45b582248a73dfe8a9c4"
format_stamp: "Formatted at 2026-08-06 22:45:21 on dist-test-slave-h3ft"
server_key: "5b8529946cf4fff58ca3c4fadb383e34"
server_key_iv: "0b58d2833cf414aa4ba978950de0123e"
server_key_version: "encryptionkey@0"
I20260806 22:45:21.170313 30538 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:45:21.171217 30556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:21.171416 30538 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:45:21.171546 30538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-0/wal
uuid: "01763e39229d45b582248a73dfe8a9c4"
format_stamp: "Formatted at 2026-08-06 22:45:21 on dist-test-slave-h3ft"
server_key: "5b8529946cf4fff58ca3c4fadb383e34"
server_key_iv: "0b58d2833cf414aa4ba978950de0123e"
server_key_version: "encryptionkey@0"
I20260806 22:45:21.171629 30538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:45:21.185726 30538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:45:21.188377 30538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:45:21.188591 30538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:45:21.188925 30538 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:45:21.189332 30538 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:45:21.189391 30538 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:21.189440 30538 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:45:21.189497 30538 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:21.197193 30538 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.193:36545
I20260806 22:45:21.197273 30717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.193:36545 every 8 connection(s)
I20260806 22:45:21.197666 30538 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
Aug 06 22:45:21 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056321, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.193@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:45:21.205790 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 30538
I20260806 22:45:21.205936 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-0/wal/instance
I20260806 22:45:21.206202 18615 external_mini_cluster.cc:1428] Setting key 71af03be46ded5dfa689eed0f112141e
I20260806 22:45:21.209905 30505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:21.198613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:56071 (local address 127.18.45.254:40873)
0806 22:45:21.198698 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:21.198702 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:21.198964 (+ 262us) server_negotiation.cc:407] Connection header received
0806 22:45:21.199709 (+ 745us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:21.199711 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:21.199728 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:21.199789 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:21.200977 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:21.201587 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:21.202716 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:21.202973 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:21.203095 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:21.205823 (+ 2728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:21.205827 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:21.205828 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:21.205844 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:21.208042 (+ 2198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:21.208524 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:21.208526 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:21.208526 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:21.208605 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:21.208965 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:21.208967 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:21.208967 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:21.209146 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:21.209226 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:21.209641 (+ 415us) server_negotiation.cc:299] Negotiation successful
0806 22:45:21.209786 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 22:45:21.210172 30720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:21.198589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:40873 (local address 127.18.45.193:56071)
0806 22:45:21.198810 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:21.198861 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:21.199568 (+ 707us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:21.199873 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:21.199877 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:21.200594 (+ 717us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:21.200862 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:21.200868 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:21.201680 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:21.201681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:21.202608 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:21.202612 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:21.203105 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:21.203106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:21.203233 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:21.203946 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:21.203957 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:21.205686 (+ 1729us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:45:21.208141 (+ 2455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:21.208150 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:21.208152 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:21.208435 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:21.208709 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:21.208710 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:21.208710 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:21.208884 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:21.209237 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:45:21.209245 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:45:21.209425 (+ 180us) client_negotiation.cc:769] Sending connection context
0806 22:45:21.209599 (+ 174us) client_negotiation.cc:240] Negotiation successful
0806 22:45:21.209842 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":125,"threads_started":1}
I20260806 22:45:21.210812 30718 heartbeater.cc:347] Connected to a master server at 127.18.45.254:40873
I20260806 22:45:21.210986 30718 heartbeater.cc:464] Registering TS with master...
I20260806 22:45:21.211267 30718 heartbeater.cc:511] Master 127.18.45.254:40873 requested a full tablet report, sending...
I20260806 22:45:21.211987 30456 ts_manager.cc:194] Registered new tserver with Master: 01763e39229d45b582248a73dfe8a9c4 (127.18.45.193:36545)
I20260806 22:45:21.212757 30456 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.45.193@KRBTEST.COM'} at 127.18.45.193:56071
WARNING: no policy specified for kudu/127.18.45.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab.
Entry for principal kudu/127.18.45.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab.
I20260806 22:45:21.240481 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.194:0
--local_ip_for_outbound_sockets=127.18.45.194
--webserver_interface=127.18.45.194
--webserver_port=0
--tserver_master_addrs=127.18.45.254:40873
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab
--principal=kudu/127.18.45.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:41485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:45:21.339658 30724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:45:21.339969 30724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:45:21.340014 30724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:45:21.341449 30724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:45:21.341528 30724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:45:21.341572 30724 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.194
I20260806 22:45:21.343349 30724 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:41485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab
--principal=kudu/127.18.45.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.45.194
--webserver_port=0
--tserver_master_addrs=127.18.45.254:40873
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30724
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.194
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:45:21.343681 30724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:45:21.343984 30724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:45:21.350122 30731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:21.350167 30732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:21.350291 30724 server_base.cc:1034] running on GCE node
W20260806 22:45:21.350113 30735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:21.350590 30724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:45:21.350906 30724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:45:21.352101 30724 hybrid_clock.cc:648] HybridClock initialized: now 1786056321352101 us; error 48 us; skew 500 ppm
Aug 06 22:45:21 dist-test-slave-h3ft krb5kdc[30386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056321, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:45:21.355234 30724 init.cc:377] Logged in from keytab as kudu/127.18.45.194@KRBTEST.COM (short username kudu)
I20260806 22:45:21.355804 30724 webserver.cc:460] Webserver started at http://127.18.45.194:34771/ using document root <none> and password file <none>
I20260806 22:45:21.356112 30724 fs_manager.cc:359] Metadata directory not provided
I20260806 22:45:21.356199 30724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:45:21.356366 30724 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:45:21.357601 30724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-1/data/instance:
uuid: "8075ddbea49b483cb6cac4faa0f4292a"
format_stamp: "Formatted at 2026-08-06 22:45:21 on dist-test-slave-h3ft"
server_key: "6d7543a63fef28b21b0d8dde218a1213"
server_key_iv: "7f42198b02dbcbffeedd161b057c49c6"
server_key_version: "encryptionkey@0"
I20260806 22:45:21.357949 30724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-1/wal/instance:
uuid: "8075ddbea49b483cb6cac4faa0f4292a"
format_stamp: "Formatted at 2026-08-06 22:45:21 on dist-test-slave-h3ft"
server_key: "6d7543a63fef28b21b0d8dde218a1213"
server_key_iv: "7f42198b02dbcbffeedd161b057c49c6"
server_key_version: "encryptionkey@0"
I20260806 22:45:21.359483 30724 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:45:21.360273 30746 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:21.360476 30724 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:45:21.360561 30724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-1/wal
uuid: "8075ddbea49b483cb6cac4faa0f4292a"
format_stamp: "Formatted at 2026-08-06 22:45:21 on dist-test-slave-h3ft"
server_key: "6d7543a63fef28b21b0d8dde218a1213"
server_key_iv: "7f42198b02dbcbffeedd161b057c49c6"
server_key_version: "encryptionkey@0"
I20260806 22:45:21.360626 30724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:45:21.383184 30724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:45:21.385998 30724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:45:21.386149 30724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:45:21.386394 30724 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:45:21.386787 30724 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:45:21.386826 30724 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:21.386874 30724 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:45:21.386904 30724 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:21.394181 30724 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.194:37491
I20260806 22:45:21.394234 30917 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.194:37491 every 8 connection(s)
I20260806 22:45:21.394661 30724 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
I20260806 22:45:21.398195 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 30724
I20260806 22:45:21.398308 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-1/wal/instance
I20260806 22:45:21.398564 18615 external_mini_cluster.cc:1428] Setting key 475f698c15c502983127a7f40ba03839
Aug 06 22:45:21 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056321, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.194@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:45:21.406507 30505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:21.395708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:60425 (local address 127.18.45.254:40873)
0806 22:45:21.395787 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:21.395789 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:21.396009 (+ 220us) server_negotiation.cc:407] Connection header received
0806 22:45:21.396646 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:21.396647 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:21.396662 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:21.396740 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:21.397892 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:21.398567 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:21.400101 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:21.400314 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:21.400397 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:21.402968 (+ 2571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:21.402970 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:21.402970 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:21.402981 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:21.404727 (+ 1746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:21.405327 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:21.405328 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:21.405328 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:21.405393 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:21.405714 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:21.405715 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:21.405715 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:21.405841 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:21.405941 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:21.406332 (+ 391us) server_negotiation.cc:299] Negotiation successful
0806 22:45:21.406423 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 22:45:21.406877 30920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:21.395637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:40873 (local address 127.18.45.194:60425)
0806 22:45:21.395846 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:21.395901 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:21.396503 (+ 602us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:21.396836 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:21.396840 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:21.397509 (+ 669us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:21.397778 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:21.397783 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:21.398714 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:21.398715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:21.400001 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:21.400004 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:21.400409 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:21.400410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:21.400573 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:21.401237 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:21.401255 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:21.402866 (+ 1611us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:45:21.404818 (+ 1952us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:21.404828 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:21.404831 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:21.405231 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:21.405464 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:21.405465 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:21.405466 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:21.405637 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:21.405938 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:45:21.405948 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:45:21.406155 (+ 207us) client_negotiation.cc:769] Sending connection context
0806 22:45:21.406305 (+ 150us) client_negotiation.cc:240] Negotiation successful
0806 22:45:21.406540 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":107,"threads_started":1}
I20260806 22:45:21.407586 30918 heartbeater.cc:347] Connected to a master server at 127.18.45.254:40873
I20260806 22:45:21.407727 30918 heartbeater.cc:464] Registering TS with master...
I20260806 22:45:21.407996 30918 heartbeater.cc:511] Master 127.18.45.254:40873 requested a full tablet report, sending...
I20260806 22:45:21.408564 30456 ts_manager.cc:194] Registered new tserver with Master: 8075ddbea49b483cb6cac4faa0f4292a (127.18.45.194:37491)
I20260806 22:45:21.409173 30456 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.45.194@KRBTEST.COM'} at 127.18.45.194:60425
WARNING: no policy specified for kudu/127.18.45.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab.
Entry for principal kudu/127.18.45.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab.
I20260806 22:45:21.435390 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.195:0
--local_ip_for_outbound_sockets=127.18.45.195
--webserver_interface=127.18.45.195
--webserver_port=0
--tserver_master_addrs=127.18.45.254:40873
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab
--principal=kudu/127.18.45.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:41485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:45:21.536728 30924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:45:21.536995 30924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:45:21.537024 30924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:45:21.538260 30924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:45:21.538300 30924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:45:21.538326 30924 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.195
I20260806 22:45:21.539973 30924 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:41485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab
--principal=kudu/127.18.45.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.45.195
--webserver_port=0
--tserver_master_addrs=127.18.45.254:40873
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30924
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.195
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:45:21.540225 30924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:45:21.540459 30924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:45:21.547279 30932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:21.547279 30938 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:21.547272 30930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:21.547686 30924 server_base.cc:1034] running on GCE node
I20260806 22:45:21.547981 30924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:45:21.548234 30924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:45:21.549440 30924 hybrid_clock.cc:648] HybridClock initialized: now 1786056321549406 us; error 46 us; skew 500 ppm
Aug 06 22:45:21 dist-test-slave-h3ft krb5kdc[30386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056321, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:45:21.553013 30924 init.cc:377] Logged in from keytab as kudu/127.18.45.195@KRBTEST.COM (short username kudu)
I20260806 22:45:21.553586 30924 webserver.cc:460] Webserver started at http://127.18.45.195:33533/ using document root <none> and password file <none>
I20260806 22:45:21.553874 30924 fs_manager.cc:359] Metadata directory not provided
I20260806 22:45:21.553921 30924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:45:21.554073 30924 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:45:21.555430 30924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-2/data/instance:
uuid: "13f5f340ee0c46868fd7ffb8053e60a7"
format_stamp: "Formatted at 2026-08-06 22:45:21 on dist-test-slave-h3ft"
server_key: "c13079331779f1cde1102b0af0d36f95"
server_key_iv: "f8017dc502d2e00c0c5f042df57f5549"
server_key_version: "encryptionkey@0"
I20260806 22:45:21.555879 30924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-2/wal/instance:
uuid: "13f5f340ee0c46868fd7ffb8053e60a7"
format_stamp: "Formatted at 2026-08-06 22:45:21 on dist-test-slave-h3ft"
server_key: "c13079331779f1cde1102b0af0d36f95"
server_key_iv: "f8017dc502d2e00c0c5f042df57f5549"
server_key_version: "encryptionkey@0"
I20260806 22:45:21.557438 30924 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:45:21.558362 30954 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:21.558634 30924 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:45:21.558743 30924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-2/wal
uuid: "13f5f340ee0c46868fd7ffb8053e60a7"
format_stamp: "Formatted at 2026-08-06 22:45:21 on dist-test-slave-h3ft"
server_key: "c13079331779f1cde1102b0af0d36f95"
server_key_iv: "f8017dc502d2e00c0c5f042df57f5549"
server_key_version: "encryptionkey@0"
I20260806 22:45:21.558828 30924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:45:21.586154 30924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:45:21.589012 30924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:45:21.589201 30924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:45:21.589524 30924 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:45:21.589926 30924 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:45:21.589989 30924 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:21.590039 30924 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:45:21.590066 30924 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:21.597375 30924 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.195:33457
I20260806 22:45:21.597425 31113 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.195:33457 every 8 connection(s)
I20260806 22:45:21.597808 30924 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
I20260806 22:45:21.603756 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 30924
I20260806 22:45:21.603965 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-2/wal/instance
I20260806 22:45:21.604231 18615 external_mini_cluster.cc:1428] Setting key eb1a53193d53dbe7cb3a0120daf945bf
Aug 06 22:45:21 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056321, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.195@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:45:21.608965 30505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:21.598706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.195:53653 (local address 127.18.45.254:40873)
0806 22:45:21.598819 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:21.598822 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:21.599194 (+ 372us) server_negotiation.cc:407] Connection header received
0806 22:45:21.599790 (+ 596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:21.599791 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:21.599805 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:21.599865 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:21.600980 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:21.601532 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:21.602678 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:21.602957 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:21.603061 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:21.605823 (+ 2762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:21.605825 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:21.605826 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:21.605837 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:21.607419 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:21.607863 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:21.607865 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:21.607865 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:21.607934 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:21.608209 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:21.608211 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:21.608211 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:21.608380 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:21.608461 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:21.608759 (+ 298us) server_negotiation.cc:299] Negotiation successful
0806 22:45:21.608862 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 22:45:21.609058 31117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:21.598668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:40873 (local address 127.18.45.195:53653)
0806 22:45:21.598923 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:21.598972 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:21.599686 (+ 714us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:21.599961 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:21.599965 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:21.600632 (+ 667us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:21.600868 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:21.600874 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:21.601639 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:21.601641 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:21.602557 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:21.602560 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:21.603095 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:21.603097 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:21.603222 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:21.603849 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:21.603858 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:21.605718 (+ 1860us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:45:21.607508 (+ 1790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:21.607515 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:21.607518 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:21.607769 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:21.608006 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:21.608007 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:21.608007 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:21.608124 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:21.608451 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:45:21.608456 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:45:21.608599 (+ 143us) client_negotiation.cc:769] Sending connection context
0806 22:45:21.608695 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 22:45:21.608843 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":125,"threads_started":1}
I20260806 22:45:21.609813 31114 heartbeater.cc:347] Connected to a master server at 127.18.45.254:40873
I20260806 22:45:21.609966 31114 heartbeater.cc:464] Registering TS with master...
I20260806 22:45:21.610185 31114 heartbeater.cc:511] Master 127.18.45.254:40873 requested a full tablet report, sending...
I20260806 22:45:21.610728 30456 ts_manager.cc:194] Registered new tserver with Master: 13f5f340ee0c46868fd7ffb8053e60a7 (127.18.45.195:33457)
I20260806 22:45:21.611269 30456 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.45.195@KRBTEST.COM'} at 127.18.45.195:53653
I20260806 22:45:21.616930 18615 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.18.45.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.45.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.45.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/krb5kdc/oryx_127.18.45.193.keytab.
Entry for principal oryx/127.18.45.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/krb5kdc/oryx_127.18.45.193.keytab.
WARNING: no policy specified for oryx/127.18.45.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.45.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.45.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/krb5kdc/oryx_127.18.45.194.keytab.
Entry for principal oryx/127.18.45.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/krb5kdc/oryx_127.18.45.194.keytab.
WARNING: no policy specified for oryx/127.18.45.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.45.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.45.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/krb5kdc/oryx_127.18.45.195.keytab.
Entry for principal oryx/127.18.45.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/krb5kdc/oryx_127.18.45.195.keytab.
I20260806 22:45:21.720024 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 30538
I20260806 22:45:21.745383 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 30724
I20260806 22:45:21.764928 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 30924
I20260806 22:45:21.778556 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 30407
I20260806 22:45:21.793220 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:40873
--webserver_interface=127.18.45.254
--webserver_port=44781
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:41485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.45.254:40873
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:45:21.895089 31127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:45:21.895385 31127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:45:21.895414 31127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:45:21.896823 31127 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:45:21.896873 31127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:45:21.896890 31127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:45:21.896904 31127 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:45:21.896919 31127 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:45:21.898763 31127 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:41485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.45.254:40873
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:40873
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.45.254
--webserver_port=44781
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31127
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:45:21.899147 31127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:45:21.899416 31127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:45:21.905821 31137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:21.905823 31139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:21.905963 31136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:21.905856 31127 server_base.cc:1034] running on GCE node
I20260806 22:45:21.906450 31127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:45:21.906855 31127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:45:21.908043 31127 hybrid_clock.cc:648] HybridClock initialized: now 1786056321908000 us; error 57 us; skew 500 ppm
Aug 06 22:45:21 dist-test-slave-h3ft krb5kdc[30386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056321, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:45:21.911646 31127 init.cc:377] Logged in from keytab as kudu/127.18.45.254@KRBTEST.COM (short username kudu)
I20260806 22:45:21.912272 31127 webserver.cc:460] Webserver started at http://127.18.45.254:44781/ using document root <none> and password file <none>
I20260806 22:45:21.912601 31127 fs_manager.cc:359] Metadata directory not provided
I20260806 22:45:21.912715 31127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:45:21.914574 31127 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:45:21.915642 31148 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:21.915887 31127 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:45:21.916055 31127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/master-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/master-0/wal
uuid: "652683bd3ead4e4b896e35b45fc31dc8"
format_stamp: "Formatted at 2026-08-06 22:45:20 on dist-test-slave-h3ft"
server_key: "92b471661d08f3ead691860047c4e7dc"
server_key_iv: "ca5ab7c9d9f28e6d287485a5d2ad3b81"
server_key_version: "encryptionkey@0"
I20260806 22:45:21.916549 31127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:45:21.930259 31127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:45:21.933816 31127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:45:21.934074 31127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:45:21.939765 31127 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.254:40873
I20260806 22:45:21.939857 31228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.254:40873 every 8 connection(s)
I20260806 22:45:21.940382 31127 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
I20260806 22:45:21.940937 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 31127
I20260806 22:45:21.941334 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.193:36545
--local_ip_for_outbound_sockets=127.18.45.193
--tserver_master_addrs=127.18.45.254:40873
--webserver_port=36015
--webserver_interface=127.18.45.193
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab
--principal=kudu/127.18.45.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:41485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.18.45.193
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/krb5kdc/oryx_127.18.45.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:45:21.942582 31231 sys_catalog.cc:263] Verifying existing consensus state
I20260806 22:45:21.943574 31231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 652683bd3ead4e4b896e35b45fc31dc8: Bootstrap starting.
I20260806 22:45:21.956768 31231 log.cc:824] T 00000000000000000000000000000000 P 652683bd3ead4e4b896e35b45fc31dc8: Log is configured to *not* fsync() on all Append() calls
I20260806 22:45:21.959707 31231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 652683bd3ead4e4b896e35b45fc31dc8: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:45:21.960093 31231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 652683bd3ead4e4b896e35b45fc31dc8: Bootstrap complete.
I20260806 22:45:21.962893 31231 raft_consensus.cc:348] T 00000000000000000000000000000000 P 652683bd3ead4e4b896e35b45fc31dc8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "652683bd3ead4e4b896e35b45fc31dc8" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 40873 } }
I20260806 22:45:21.963303 31231 raft_consensus.cc:729] T 00000000000000000000000000000000 P 652683bd3ead4e4b896e35b45fc31dc8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652683bd3ead4e4b896e35b45fc31dc8, State: Initialized, Role: FOLLOWER
I20260806 22:45:21.963572 31231 consensus_queue.cc:260] T 00000000000000000000000000000000 P 652683bd3ead4e4b896e35b45fc31dc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "652683bd3ead4e4b896e35b45fc31dc8" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 40873 } }
I20260806 22:45:21.963688 31231 raft_consensus.cc:388] T 00000000000000000000000000000000 P 652683bd3ead4e4b896e35b45fc31dc8 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:45:21.963749 31231 raft_consensus.cc:482] T 00000000000000000000000000000000 P 652683bd3ead4e4b896e35b45fc31dc8 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:45:21.963805 31231 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 652683bd3ead4e4b896e35b45fc31dc8 [term 1 FOLLOWER]: Advancing to term 2
I20260806 22:45:21.965113 31231 raft_consensus.cc:504] T 00000000000000000000000000000000 P 652683bd3ead4e4b896e35b45fc31dc8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "652683bd3ead4e4b896e35b45fc31dc8" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 40873 } }
I20260806 22:45:21.965258 31231 leader_election.cc:302] T 00000000000000000000000000000000 P 652683bd3ead4e4b896e35b45fc31dc8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 652683bd3ead4e4b896e35b45fc31dc8; no voters:
I20260806 22:45:21.965529 31231 leader_election.cc:288] T 00000000000000000000000000000000 P 652683bd3ead4e4b896e35b45fc31dc8 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 22:45:21.965592 31239 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 652683bd3ead4e4b896e35b45fc31dc8 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 22:45:21.965745 31239 raft_consensus.cc:686] T 00000000000000000000000000000000 P 652683bd3ead4e4b896e35b45fc31dc8 [term 2 LEADER]: Becoming Leader. State: Replica: 652683bd3ead4e4b896e35b45fc31dc8, State: Running, Role: LEADER
I20260806 22:45:21.965895 31231 sys_catalog.cc:564] T 00000000000000000000000000000000 P 652683bd3ead4e4b896e35b45fc31dc8 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:45:21.965878 31239 consensus_queue.cc:237] T 00000000000000000000000000000000 P 652683bd3ead4e4b896e35b45fc31dc8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "652683bd3ead4e4b896e35b45fc31dc8" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 40873 } }
I20260806 22:45:21.966228 31239 sys_catalog.cc:455] T 00000000000000000000000000000000 P 652683bd3ead4e4b896e35b45fc31dc8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 652683bd3ead4e4b896e35b45fc31dc8. Latest consensus state: current_term: 2 leader_uuid: "652683bd3ead4e4b896e35b45fc31dc8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "652683bd3ead4e4b896e35b45fc31dc8" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 40873 } } }
I20260806 22:45:21.966317 31239 sys_catalog.cc:458] T 00000000000000000000000000000000 P 652683bd3ead4e4b896e35b45fc31dc8 [sys.catalog]: This master's current role is: LEADER
I20260806 22:45:21.966234 31240 sys_catalog.cc:455] T 00000000000000000000000000000000 P 652683bd3ead4e4b896e35b45fc31dc8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "652683bd3ead4e4b896e35b45fc31dc8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "652683bd3ead4e4b896e35b45fc31dc8" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 40873 } } }
I20260806 22:45:21.966540 31240 sys_catalog.cc:458] T 00000000000000000000000000000000 P 652683bd3ead4e4b896e35b45fc31dc8 [sys.catalog]: This master's current role is: LEADER
I20260806 22:45:21.966629 31243 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:45:21.967515 31243 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:45:21.967772 31243 catalog_manager.cc:1259] Loaded cluster ID: 6116af451d3e42fcbb37cbe429c2f7f1
I20260806 22:45:21.967974 31243 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:45:21.969245 31243 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:45:21.969455 31243 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 652683bd3ead4e4b896e35b45fc31dc8: Loaded TSK: 0
I20260806 22:45:21.969694 31243 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 22:45:22.048827 31234 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:45:22.049208 31234 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:45:22.049261 31234 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:45:22.050693 31234 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:45:22.050772 31234 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:45:22.050827 31234 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.193
I20260806 22:45:22.052716 31234 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:41485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/krb5kdc/oryx_127.18.45.193.keytab
--principal=oryx/127.18.45.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.193:36545
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.45.193
--webserver_port=36015
--tserver_master_addrs=127.18.45.254:40873
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31234
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.193
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:45:22.053066 31234 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:45:22.053400 31234 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:45:22.059688 31269 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:22.059705 31263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:22.059722 31271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:22.060417 31234 server_base.cc:1034] running on GCE node
I20260806 22:45:22.060734 31234 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:45:22.061056 31234 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:45:22.062244 31234 hybrid_clock.cc:648] HybridClock initialized: now 1786056322062229 us; error 36 us; skew 500 ppm
Aug 06 22:45:22 dist-test-slave-h3ft krb5kdc[30386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056322, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.45.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:45:22.065693 31234 init.cc:377] Logged in from keytab as oryx/127.18.45.193@KRBTEST.COM (short username oryx)
I20260806 22:45:22.066332 31234 webserver.cc:460] Webserver started at http://127.18.45.193:36015/ using document root <none> and password file <none>
I20260806 22:45:22.066747 31234 fs_manager.cc:359] Metadata directory not provided
I20260806 22:45:22.066849 31234 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:45:22.068712 31234 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:45:22.069495 31281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:22.069662 31234 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:45:22.069764 31234 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-0/wal
uuid: "01763e39229d45b582248a73dfe8a9c4"
format_stamp: "Formatted at 2026-08-06 22:45:21 on dist-test-slave-h3ft"
server_key: "5b8529946cf4fff58ca3c4fadb383e34"
server_key_iv: "0b58d2833cf414aa4ba978950de0123e"
server_key_version: "encryptionkey@0"
I20260806 22:45:22.070226 31234 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:45:22.097033 31234 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:45:22.101876 31234 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:45:22.102077 31234 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:45:22.102406 31234 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:45:22.102841 31234 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:45:22.102888 31234 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:22.102926 31234 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:45:22.102962 31234 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:22.112308 31234 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.193:36545
I20260806 22:45:22.112502 31455 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.193:36545 every 8 connection(s)
I20260806 22:45:22.113190 31234 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
I20260806 22:45:22.120838 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 31234
I20260806 22:45:22.121263 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.194:37491
--local_ip_for_outbound_sockets=127.18.45.194
--tserver_master_addrs=127.18.45.254:40873
--webserver_port=34771
--webserver_interface=127.18.45.194
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab
--principal=kudu/127.18.45.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:41485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.18.45.194
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/krb5kdc/oryx_127.18.45.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 22:45:22 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.193@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:45:22 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.193@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:45:22.128844 31460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:22.114331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:40873 (local address 127.18.45.193:58913)
0806 22:45:22.114874 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:22.114938 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:22.116032 (+ 1094us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:22.116839 (+ 807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:22.116844 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:22.117944 (+ 1100us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:22.118316 (+ 372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:22.118325 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:22.120214 (+ 1889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:22.120215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:22.122633 (+ 2418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:22.122639 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:22.123772 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:22.123774 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:22.123966 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:22.124878 (+ 912us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:22.124892 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:22.128513 (+ 3621us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.45.254:40873: Server oryx/127.18.45.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":114,"threads_started":1}
W20260806 22:45:22.129040 31461 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:45:22.114420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:58913 (local address 127.18.45.254:40873)
0806 22:45:22.114745 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:22.114757 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:22.115122 (+ 365us) server_negotiation.cc:407] Connection header received
0806 22:45:22.116333 (+ 1211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:22.116345 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:22.116423 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:22.116571 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:22.118613 (+ 2042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:22.120066 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:22.122771 (+ 2705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:22.123663 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:22.123775 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:22.128763 (+ 4988us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.45.193:58913: BlockingRecv error: recv got EOF from 127.18.45.193:58913 (error 108)
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":144,"threads_started":1}
W20260806 22:45:22.129560 31456 heartbeater.cc:644] Failed to heartbeat to 127.18.45.254:40873 (0 consecutive failures): Not authorized: Failed to ping master at 127.18.45.254:40873: Client connection negotiation failed: client connection to 127.18.45.254:40873: Server oryx/127.18.45.254@KRBTEST.COM not found in Kerberos database
Aug 06 22:45:22 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.193@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:45:22 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.193@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:45:22.136602 31460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:22.129875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:40873 (local address 127.18.45.193:34125)
0806 22:45:22.129997 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:22.130013 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:22.130065 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:22.130232 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:22.130233 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:22.130372 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:22.130446 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:22.130448 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:22.131611 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:22.131613 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:22.132298 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:22.132302 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:22.132972 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:22.132973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:22.133047 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:22.133762 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:22.133767 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:22.136490 (+ 2723us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.45.254:40873: Server oryx/127.18.45.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":100}
W20260806 22:45:22.136695 31461 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:45:22.129947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:34125 (local address 127.18.45.254:40873)
0806 22:45:22.130015 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:22.130019 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:22.130053 (+ 34us) server_negotiation.cc:407] Connection header received
0806 22:45:22.130123 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:22.130124 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:22.130147 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:22.130231 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:22.130551 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:22.131497 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:22.132471 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:22.132851 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:22.132956 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:22.136608 (+ 3652us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.45.193:34125: BlockingRecv error: recv got EOF from 127.18.45.193:34125 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 06 22:45:22 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.193@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:45:22 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.193@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:45:22.144110 31460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:22.137188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:40873 (local address 127.18.45.193:44175)
0806 22:45:22.137244 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:22.137255 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:22.137309 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:22.137500 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:22.137502 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:22.137719 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:22.137785 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:22.137787 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:22.138967 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:22.138968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:22.139934 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:22.139938 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:22.140471 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:22.140472 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:22.140560 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:22.141175 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:22.141183 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:22.143965 (+ 2782us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.45.254:40873: Server oryx/127.18.45.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":35}
W20260806 22:45:22.144181 31461 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:45:22.137291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:44175 (local address 127.18.45.254:40873)
0806 22:45:22.137341 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:22.137343 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:22.137356 (+ 13us) server_negotiation.cc:407] Connection header received
0806 22:45:22.137390 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:22.137391 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:22.137407 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:22.137481 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:22.137879 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:22.138787 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:22.140027 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:22.140377 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:22.140452 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:22.144101 (+ 3649us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.45.193:44175: BlockingRecv error: recv got EOF from 127.18.45.193:44175 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260806 22:45:22.144840 31456 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 22:45:22.226729 31462 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:45:22.226976 31462 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:45:22.227020 31462 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:45:22.228377 31462 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:45:22.228423 31462 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:45:22.228452 31462 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.194
I20260806 22:45:22.230015 31462 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:41485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/krb5kdc/oryx_127.18.45.194.keytab
--principal=oryx/127.18.45.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.194:37491
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.45.194
--webserver_port=34771
--tserver_master_addrs=127.18.45.254:40873
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31462
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.194
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:45:22.230327 31462 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:45:22.230625 31462 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:45:22.236567 31469 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:22.236583 31472 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:22.236658 31474 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:22.237229 31462 server_base.cc:1034] running on GCE node
I20260806 22:45:22.237460 31462 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:45:22.237753 31462 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:45:22.241004 31462 hybrid_clock.cc:648] HybridClock initialized: now 1786056322240970 us; error 48 us; skew 500 ppm
Aug 06 22:45:22 dist-test-slave-h3ft krb5kdc[30386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056322, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.45.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:45:22.244575 31462 init.cc:377] Logged in from keytab as oryx/127.18.45.194@KRBTEST.COM (short username oryx)
I20260806 22:45:22.245136 31462 webserver.cc:460] Webserver started at http://127.18.45.194:34771/ using document root <none> and password file <none>
I20260806 22:45:22.245458 31462 fs_manager.cc:359] Metadata directory not provided
I20260806 22:45:22.245566 31462 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:45:22.247308 31462 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 22:45:22.248168 31483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:22.248459 31462 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:45:22.248581 31462 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-1/wal
uuid: "8075ddbea49b483cb6cac4faa0f4292a"
format_stamp: "Formatted at 2026-08-06 22:45:21 on dist-test-slave-h3ft"
server_key: "6d7543a63fef28b21b0d8dde218a1213"
server_key_iv: "7f42198b02dbcbffeedd161b057c49c6"
server_key_version: "encryptionkey@0"
I20260806 22:45:22.249140 31462 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:45:22.264355 31462 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:45:22.267417 31462 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:45:22.267620 31462 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:45:22.267951 31462 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:45:22.268347 31462 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:45:22.268405 31462 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:22.268446 31462 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:45:22.268494 31462 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:22.276048 31462 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.194:37491
I20260806 22:45:22.276105 31649 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.194:37491 every 8 connection(s)
I20260806 22:45:22.276527 31462 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
I20260806 22:45:22.281416 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 31462
I20260806 22:45:22.281819 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.195:33457
--local_ip_for_outbound_sockets=127.18.45.195
--tserver_master_addrs=127.18.45.254:40873
--webserver_port=33533
--webserver_interface=127.18.45.195
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab
--principal=kudu/127.18.45.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:41485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.18.45.195
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/krb5kdc/oryx_127.18.45.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 22:45:22 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.194@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:45:22 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.194@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
W20260806 22:45:22.286659 31461 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:45:22.277459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:59531 (local address 127.18.45.254:40873)
0806 22:45:22.277537 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:22.277539 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:22.277850 (+ 311us) server_negotiation.cc:407] Connection header received
0806 22:45:22.278549 (+ 699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:22.278551 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:22.278574 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:22.278644 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:22.279871 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:22.280423 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:22.281803 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:22.282152 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:22.282242 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:22.286589 (+ 4347us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.45.194:59531: BlockingRecv error: recv got EOF from 127.18.45.194:59531 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 22:45:22.286731 31652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:22.277451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:40873 (local address 127.18.45.194:59531)
0806 22:45:22.277680 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:22.277726 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:22.278418 (+ 692us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:22.278760 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:22.278765 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:22.279514 (+ 749us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:22.279762 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:22.279768 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:22.280527 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:22.280530 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:22.281694 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:22.281699 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:22.282262 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:22.282263 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:22.282433 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:22.283272 (+ 839us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:22.283284 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:22.286485 (+ 3201us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.45.254:40873: Server oryx/127.18.45.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":119,"threads_started":1}
W20260806 22:45:22.287034 31650 heartbeater.cc:644] Failed to heartbeat to 127.18.45.254:40873 (0 consecutive failures): Not authorized: Failed to ping master at 127.18.45.254:40873: Client connection negotiation failed: client connection to 127.18.45.254:40873: Server oryx/127.18.45.254@KRBTEST.COM not found in Kerberos database
Aug 06 22:45:22 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.194@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:45:22 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.194@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:45:22.292160 31652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:22.287304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:40873 (local address 127.18.45.194:58507)
0806 22:45:22.287359 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:22.287371 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:22.287441 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:22.287584 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:22.287585 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:22.287733 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:22.287785 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:22.287786 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:22.288540 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:22.288541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:22.289076 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:22.289078 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:22.289468 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:22.289469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:22.289557 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:22.290160 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:22.290166 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:22.292059 (+ 1893us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.45.254:40873: Server oryx/127.18.45.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":34}
W20260806 22:45:22.292227 31461 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:45:22.287389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:58507 (local address 127.18.45.254:40873)
0806 22:45:22.287461 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:22.287463 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:22.287473 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:45:22.287506 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:22.287507 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:22.287518 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:22.287576 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:22.287872 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:22.288456 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:22.289161 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:22.289378 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:22.289468 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:22.292152 (+ 2684us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.45.194:58507: BlockingRecv error: recv got EOF from 127.18.45.194:58507 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 06 22:45:22 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.194@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:45:22 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.194@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:45:22.297300 31652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:22.292494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:40873 (local address 127.18.45.194:42377)
0806 22:45:22.292624 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:22.292636 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:22.292687 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:22.292847 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:22.292848 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:22.292985 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:22.293017 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:22.293018 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:22.293663 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:22.293664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:22.294164 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:22.294166 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:22.294526 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:22.294527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:22.294581 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:22.295214 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:22.295220 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:22.297195 (+ 1975us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.45.254:40873: Server oryx/127.18.45.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113}
W20260806 22:45:22.297365 31461 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:45:22.292556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:42377 (local address 127.18.45.254:40873)
0806 22:45:22.292623 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:22.292625 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:22.292670 (+ 45us) server_negotiation.cc:407] Connection header received
0806 22:45:22.292750 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:22.292751 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:22.292761 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:22.292826 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:22.293090 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:22.293577 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:22.294258 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:22.294454 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:22.294529 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:22.297286 (+ 2757us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.45.194:42377: BlockingRecv error: recv got EOF from 127.18.45.194:42377 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260806 22:45:22.297514 31650 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 22:45:22.385973 31654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:45:22.386294 31654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:45:22.386337 31654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:45:22.387687 31654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:45:22.387782 31654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:45:22.387842 31654 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.195
I20260806 22:45:22.389599 31654 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:41485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/krb5kdc/oryx_127.18.45.195.keytab
--principal=oryx/127.18.45.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.195:33457
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.45.195
--webserver_port=33533
--tserver_master_addrs=127.18.45.254:40873
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31654
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.195
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:45:22.389946 31654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:45:22.390264 31654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:45:22.396317 31670 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:22.396287 31661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:22.396644 31654 server_base.cc:1034] running on GCE node
W20260806 22:45:22.396317 31664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:22.397029 31654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:45:22.397354 31654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:45:22.398556 31654 hybrid_clock.cc:648] HybridClock initialized: now 1786056322398539 us; error 29 us; skew 500 ppm
Aug 06 22:45:22 dist-test-slave-h3ft krb5kdc[30386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056322, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.45.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:45:22.401733 31654 init.cc:377] Logged in from keytab as oryx/127.18.45.195@KRBTEST.COM (short username oryx)
I20260806 22:45:22.402315 31654 webserver.cc:460] Webserver started at http://127.18.45.195:33533/ using document root <none> and password file <none>
I20260806 22:45:22.402601 31654 fs_manager.cc:359] Metadata directory not provided
I20260806 22:45:22.402706 31654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:45:22.404426 31654 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:45:22.405258 31680 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:22.405524 31654 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:45:22.405651 31654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-2/wal
uuid: "13f5f340ee0c46868fd7ffb8053e60a7"
format_stamp: "Formatted at 2026-08-06 22:45:21 on dist-test-slave-h3ft"
server_key: "c13079331779f1cde1102b0af0d36f95"
server_key_iv: "f8017dc502d2e00c0c5f042df57f5549"
server_key_version: "encryptionkey@0"
I20260806 22:45:22.406113 31654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:45:22.431545 31654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:45:22.434546 31654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:45:22.434751 31654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:45:22.435125 31654 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:45:22.435596 31654 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:45:22.435676 31654 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:22.435727 31654 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:45:22.435753 31654 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:22.443428 31654 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.195:33457
I20260806 22:45:22.443468 31839 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.195:33457 every 8 connection(s)
I20260806 22:45:22.443938 31654 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
I20260806 22:45:22.451153 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 31654
Aug 06 22:45:22 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.195@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:45:22 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.195@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
W20260806 22:45:22.453559 31461 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:45:22.444938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.195:56113 (local address 127.18.45.254:40873)
0806 22:45:22.445038 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:22.445040 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:22.445225 (+ 185us) server_negotiation.cc:407] Connection header received
0806 22:45:22.445913 (+ 688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:22.445916 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:22.445934 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:22.445995 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:22.447251 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:22.447802 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:22.448987 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:22.449198 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:22.449296 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:22.453486 (+ 4190us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.45.195:56113: BlockingRecv error: recv got EOF from 127.18.45.195:56113 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 22:45:22.453678 31842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:22.444837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:40873 (local address 127.18.45.195:56113)
0806 22:45:22.445049 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:22.445112 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:22.445762 (+ 650us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:22.446100 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:22.446104 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:22.446848 (+ 744us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:22.447125 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:22.447131 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:22.447931 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:22.447933 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:22.448880 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:22.448883 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:22.449321 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:22.449322 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:22.449472 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:22.450118 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:22.450128 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:22.453401 (+ 3273us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.45.254:40873: Server oryx/127.18.45.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":87,"threads_started":1}
W20260806 22:45:22.454020 31840 heartbeater.cc:644] Failed to heartbeat to 127.18.45.254:40873 (0 consecutive failures): Not authorized: Failed to ping master at 127.18.45.254:40873: Client connection negotiation failed: client connection to 127.18.45.254:40873: Server oryx/127.18.45.254@KRBTEST.COM not found in Kerberos database
Aug 06 22:45:22 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.195@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:45:22 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.195@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:45:22.460332 31842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:22.454429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:40873 (local address 127.18.45.195:36701)
0806 22:45:22.454492 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:22.454505 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:22.454560 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:22.454720 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:22.454721 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:22.454895 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:22.454962 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:22.454965 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:22.455890 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:22.455891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:22.456459 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:22.456462 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:22.456886 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:22.456888 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:22.456978 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:22.457509 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:22.457516 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:22.460213 (+ 2697us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.45.254:40873: Server oryx/127.18.45.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":39}
W20260806 22:45:22.460417 31461 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:45:22.454506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.195:36701 (local address 127.18.45.254:40873)
0806 22:45:22.454577 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:22.454579 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:22.454589 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:45:22.454617 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:22.454618 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:22.454632 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:22.454713 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:22.455103 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:22.455771 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:22.456573 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:22.456781 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:22.456880 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:22.460315 (+ 3435us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.45.195:36701: BlockingRecv error: recv got EOF from 127.18.45.195:36701 (error 108)
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":40}
I20260806 22:45:22.461689 31845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:22.451824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54268 (local address 127.18.45.254:40873)
0806 22:45:22.452053 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:22.452055 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:22.452065 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:45:22.452092 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:22.452093 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:22.452109 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:22.452198 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:22.453260 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:22.454008 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:22.455233 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:22.455485 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:22.455569 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:22.457160 (+ 1591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:22.457167 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:22.457168 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:22.457193 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:22.459571 (+ 2378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:22.460121 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:22.460126 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:22.460127 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:22.460259 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:22.460715 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:22.460716 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:22.460716 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:22.461108 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:22.461293 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:22.461428 (+ 135us) server_negotiation.cc:299] Negotiation successful
0806 22:45:22.461569 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":148,"threads_started":1}
Aug 06 22:45:22 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.195@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:45:22 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.195@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:45:22.465469 31842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:22.460694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:40873 (local address 127.18.45.195:34317)
0806 22:45:22.460752 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:22.460766 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:22.460818 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:22.460983 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:22.460984 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:22.461138 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:22.461178 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:22.461180 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:22.462056 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:22.462057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:22.462761 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:22.462764 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:22.463207 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:22.463208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:22.463259 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:22.463646 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:22.463650 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:22.465402 (+ 1752us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.45.254:40873: Server oryx/127.18.45.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":37}
W20260806 22:45:22.465721 31840 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 22:45:22.465940 31461 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:45:22.460770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.195:34317 (local address 127.18.45.254:40873)
0806 22:45:22.460844 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:22.460847 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:22.460858 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:45:22.460883 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:22.460883 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:22.460896 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:22.460977 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:22.461265 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:22.461981 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:22.462840 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:22.463146 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:22.463205 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:22.465868 (+ 2663us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.45.195:34317: BlockingRecv error: recv got EOF from 127.18.45.195:34317 (error 108)
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":44}
Aug 06 22:45:23 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.193@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:45:23 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.193@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:45:23.151746 31848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:23.145525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:40873 (local address 127.18.45.193:46507)
0806 22:45:23.145735 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:23.145760 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:23.145843 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:23.146154 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:23.146155 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:23.146416 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:23.146500 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:23.146503 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:23.147338 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:23.147339 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:23.148029 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:23.148032 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:23.148411 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:23.148412 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:23.148462 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:23.149084 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:23.149092 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:23.151653 (+ 2561us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.45.254:40873: Server oryx/127.18.45.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":131,"threads_started":1}
W20260806 22:45:23.151906 31850 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:45:23.145622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:46507 (local address 127.18.45.254:40873)
0806 22:45:23.145789 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:23.145791 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:23.145820 (+ 29us) server_negotiation.cc:407] Connection header received
0806 22:45:23.145974 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:23.145976 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:23.145995 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:23.146099 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:23.146601 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:23.147234 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:23.148121 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:23.148335 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:23.148396 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:23.151775 (+ 3379us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.45.193:46507: BlockingRecv error: recv got EOF from 127.18.45.193:46507 (error 108)
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":99,"threads_started":1}
Aug 06 22:45:23 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.194@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:45:23 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.194@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:45:23.304294 31853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:23.298304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:40873 (local address 127.18.45.194:58127)
0806 22:45:23.298549 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:23.298562 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:23.298669 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:23.298892 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:23.298894 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:23.299214 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:23.299286 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:23.299289 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:23.300081 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:23.300082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:23.300686 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:23.300687 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:23.301122 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:23.301123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:23.301181 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:23.301845 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:23.301852 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:23.304200 (+ 2348us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.45.254:40873: Server oryx/127.18.45.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":144,"threads_started":1}
W20260806 22:45:23.304404 31850 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:45:23.298420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:58127 (local address 127.18.45.254:40873)
0806 22:45:23.298523 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:23.298526 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:23.298622 (+ 96us) server_negotiation.cc:407] Connection header received
0806 22:45:23.298750 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:23.298751 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:23.298769 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:23.298850 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:23.299389 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:23.299991 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:23.300774 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:23.301015 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:23.301080 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:23.304321 (+ 3241us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.45.194:58127: BlockingRecv error: recv got EOF from 127.18.45.194:58127 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 06 22:45:23 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.195@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:45:23 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.195@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:45:23.472443 31854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:23.466500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:40873 (local address 127.18.45.195:39779)
0806 22:45:23.466730 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:23.466743 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:23.466829 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:23.467100 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:23.467101 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:23.467469 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:23.467586 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:23.467589 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:23.468445 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:23.468446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:23.469083 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:23.469085 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:23.469476 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:23.469477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:23.469525 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:23.470108 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:23.470114 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:23.472350 (+ 2236us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.45.254:40873: Server oryx/127.18.45.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":115,"threads_started":1}
W20260806 22:45:23.472648 31850 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:45:23.466592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.195:39779 (local address 127.18.45.254:40873)
0806 22:45:23.466722 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:23.466725 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:23.466804 (+ 79us) server_negotiation.cc:407] Connection header received
0806 22:45:23.466917 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:23.466918 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:23.466974 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:23.467069 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:23.467678 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:23.468357 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:23.469196 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:23.469399 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:23.469465 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:23.472476 (+ 3011us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.45.195:39779: BlockingRecv error: recv got EOF from 127.18.45.195:39779 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Aug 06 22:45:24 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.193@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:45:24 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.193@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:45:24.158437 31855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:24.152554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:40873 (local address 127.18.45.193:41697)
0806 22:45:24.152785 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:24.152801 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:24.152895 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:24.153103 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:24.153105 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:24.153353 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:24.153437 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:24.153440 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:24.154224 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:24.154225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:24.154863 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:24.154865 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:24.155308 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:24.155309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:24.155370 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:24.156038 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:24.156044 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:24.158346 (+ 2302us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.45.254:40873: Server oryx/127.18.45.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":99,"threads_started":1}
W20260806 22:45:24.158609 31856 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:45:24.152717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:41697 (local address 127.18.45.254:40873)
0806 22:45:24.152878 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:24.152880 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:24.152891 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:45:24.152954 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:24.152955 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:24.152966 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:24.153069 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:24.153546 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:24.154137 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:24.154948 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:24.155227 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:24.155287 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:24.158473 (+ 3186us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.45.193:41697: BlockingRecv error: recv got EOF from 127.18.45.193:41697 (error 108)
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":95,"threads_started":1}
Aug 06 22:45:24 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.194@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:45:24 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.194@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:45:24.311039 31857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:24.305041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:40873 (local address 127.18.45.194:41769)
0806 22:45:24.305230 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:24.305252 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:24.305346 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:24.305554 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:24.305556 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:24.305799 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:24.305871 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:24.305873 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:24.306883 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:24.306884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:24.307551 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:24.307553 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:24.308033 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:24.308034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:24.308086 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:24.308724 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:24.308730 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:24.310925 (+ 2195us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.45.254:40873: Server oryx/127.18.45.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":86,"threads_started":1}
W20260806 22:45:24.311295 31856 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:45:24.305173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:41769 (local address 127.18.45.254:40873)
0806 22:45:24.305294 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:24.305297 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:24.305314 (+ 17us) server_negotiation.cc:407] Connection header received
0806 22:45:24.305425 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:24.305426 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:24.305442 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:24.305508 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:24.306009 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:24.306773 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:24.307678 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:24.307911 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:24.308030 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:24.311208 (+ 3178us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.45.194:41769: BlockingRecv error: recv got EOF from 127.18.45.194:41769 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 06 22:45:24 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.195@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:45:24 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.195@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:45:24.484036 31858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:24.473284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:40873 (local address 127.18.45.195:39157)
0806 22:45:24.473569 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:24.473582 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:24.473650 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:24.473811 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:24.473812 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:24.474044 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:24.474115 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:24.474117 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:24.475056 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:24.475058 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:24.475825 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:24.475828 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:24.476201 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:24.476202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:24.476245 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:24.480977 (+ 4732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:24.480983 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:24.483939 (+ 2956us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.45.254:40873: Server oryx/127.18.45.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":190,"threads_started":1}
W20260806 22:45:24.484259 31856 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:45:24.473536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.195:39157 (local address 127.18.45.254:40873)
0806 22:45:24.473614 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:24.473616 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:24.473650 (+ 34us) server_negotiation.cc:407] Connection header received
0806 22:45:24.473721 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:24.473722 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:24.473738 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:24.473785 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:24.474205 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:24.474950 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:24.475926 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:24.476130 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:24.476190 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:24.484145 (+ 7955us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.45.195:39157: BlockingRecv error: recv got EOF from 127.18.45.195:39157 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 06 22:45:25 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.193@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:45:25 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.193@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:45:25.165637 31859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:25.159144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:40873 (local address 127.18.45.193:49341)
0806 22:45:25.159365 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:25.159378 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:25.159450 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:25.159639 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:25.159640 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:25.159896 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:25.159970 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:25.159972 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:25.161279 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:25.161281 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:25.162007 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:25.162012 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:25.162497 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:25.162498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:25.162562 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:25.163344 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:25.163349 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:25.165558 (+ 2209us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.45.254:40873: Server oryx/127.18.45.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":126,"threads_started":1}
W20260806 22:45:25.165891 31860 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:45:25.159267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:49341 (local address 127.18.45.254:40873)
0806 22:45:25.159438 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:25.159440 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:25.159452 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:45:25.159527 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:25.159528 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:25.159541 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:25.159604 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:25.160099 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:25.161160 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:25.162169 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:25.162389 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:25.162452 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:25.165786 (+ 3334us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.45.193:49341: BlockingRecv error: recv got EOF from 127.18.45.193:49341 (error 108)
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":102,"threads_started":1}
Aug 06 22:45:25 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.194@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:45:25 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.194@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:45:25.317788 31862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:25.311606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:40873 (local address 127.18.45.194:42457)
0806 22:45:25.311780 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:25.311800 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:25.311872 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:25.312131 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:25.312133 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:25.312384 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:25.312455 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:25.312458 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:25.313302 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:25.313304 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:25.313983 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:25.313986 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:25.314405 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:25.314406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:25.314484 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:25.315217 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:25.315224 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:25.317697 (+ 2473us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.45.254:40873: Server oryx/127.18.45.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":105,"threads_started":1}
W20260806 22:45:25.317973 31860 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:45:25.311739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:42457 (local address 127.18.45.254:40873)
0806 22:45:25.311814 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:25.311817 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:25.311865 (+ 48us) server_negotiation.cc:407] Connection header received
0806 22:45:25.311944 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:25.311944 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:25.311994 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:25.312062 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:25.312560 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:25.313188 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:25.314095 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:25.314319 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:25.314386 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:25.317852 (+ 3466us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.45.194:42457: BlockingRecv error: recv got EOF from 127.18.45.194:42457 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 06 22:45:25 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.195@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:45:25 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.195@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:45:25.491091 31864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:25.484807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:40873 (local address 127.18.45.195:44307)
0806 22:45:25.485035 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:25.485050 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:25.485129 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:25.485350 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:25.485352 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:25.485616 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:25.485692 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:25.485694 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:25.486450 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:25.486452 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:25.487113 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:25.487117 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:25.487649 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:25.487650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:25.487734 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:25.488639 (+ 905us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:25.488645 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:25.490930 (+ 2285us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.45.254:40873: Server oryx/127.18.45.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":138,"threads_started":1}
W20260806 22:45:25.491200 31860 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:45:25.484970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.195:44307 (local address 127.18.45.254:40873)
0806 22:45:25.485106 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:25.485108 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:25.485120 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:45:25.485196 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:25.485196 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:25.485211 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:25.485277 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:25.485783 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:25.486351 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:25.487262 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:25.487522 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:25.487643 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:25.491116 (+ 3473us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.45.195:44307: BlockingRecv error: recv got EOF from 127.18.45.195:44307 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
Aug 06 22:45:26 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.193@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:45:26 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.193@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:45:26.171892 31866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:26.166230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:40873 (local address 127.18.45.193:46999)
0806 22:45:26.166482 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:26.166497 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:26.166574 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:26.166768 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:26.166769 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:26.167085 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:26.167157 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:26.167159 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:26.167986 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:26.167987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:26.168558 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:26.168560 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:26.168960 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:26.168961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:26.169029 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:26.169586 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:26.169592 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:26.171817 (+ 2225us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.45.254:40873: Server oryx/127.18.45.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":158,"threads_started":1}
W20260806 22:45:26.172201 31867 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:45:26.166348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:46999 (local address 127.18.45.254:40873)
0806 22:45:26.166538 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:26.166541 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:26.166587 (+ 46us) server_negotiation.cc:407] Connection header received
0806 22:45:26.166626 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:26.166627 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:26.166643 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:26.166752 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:26.167258 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:26.167900 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:26.168646 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:26.168868 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:26.168961 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:26.171984 (+ 3023us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.45.193:46999: BlockingRecv error: recv got EOF from 127.18.45.193:46999 (error 108)
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":123,"threads_started":1}
Aug 06 22:45:26 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.194@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:45:26 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.194@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:45:26.324007 31868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:26.318419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:40873 (local address 127.18.45.194:42447)
0806 22:45:26.318613 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:26.318626 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:26.318696 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:26.318858 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:26.318859 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:26.319113 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:26.319184 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:26.319186 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:26.319945 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:26.319946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:26.320519 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:26.320521 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:26.320924 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:26.320925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:26.321030 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:26.321576 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:26.321582 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:26.323883 (+ 2301us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.45.254:40873: Server oryx/127.18.45.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":115,"threads_started":1}
W20260806 22:45:26.324095 31867 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:45:26.318518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:42447 (local address 127.18.45.254:40873)
0806 22:45:26.318601 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:26.318603 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:26.318674 (+ 71us) server_negotiation.cc:407] Connection header received
0806 22:45:26.318747 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:26.318748 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:26.318761 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:26.318844 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:26.319245 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:26.319848 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:26.320592 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:26.320835 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:26.320890 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:26.323985 (+ 3095us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.45.194:42447: BlockingRecv error: recv got EOF from 127.18.45.194:42447 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 06 22:45:26 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.195@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:45:26 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.195@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:45:26.497607 31869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:26.491863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:40873 (local address 127.18.45.195:41781)
0806 22:45:26.492100 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:26.492112 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:26.492188 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:26.492399 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:26.492401 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:26.492649 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:26.492721 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:26.492723 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:26.493660 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:26.493661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:26.494286 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:26.494289 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:26.494678 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:26.494679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:26.494754 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:26.495427 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:26.495432 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:26.497484 (+ 2052us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.45.254:40873: Server oryx/127.18.45.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":144,"threads_started":1}
W20260806 22:45:26.497651 31867 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:45:26.491970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.195:41781 (local address 127.18.45.254:40873)
0806 22:45:26.492075 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:26.492077 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:26.492180 (+ 103us) server_negotiation.cc:407] Connection header received
0806 22:45:26.492251 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:26.492252 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:26.492269 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:26.492336 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:26.492799 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:26.493568 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:26.494392 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:26.494600 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:26.494678 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:26.497578 (+ 2900us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.45.195:41781: BlockingRecv error: recv got EOF from 127.18.45.195:41781 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 06 22:45:27 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.193@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:45:27 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.193@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:45:27.178428 31870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:27.172609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:40873 (local address 127.18.45.193:60135)
0806 22:45:27.172835 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:27.172849 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:27.172939 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:27.173117 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:27.173119 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:27.173344 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:27.173464 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:27.173468 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:27.174484 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:27.174485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:27.175169 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:27.175173 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:27.175543 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:27.175544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:27.175590 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:27.176261 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:27.176266 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:27.178339 (+ 2073us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.45.254:40873: Server oryx/127.18.45.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":110,"threads_started":1}
W20260806 22:45:27.178588 31871 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:45:27.172694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:60135 (local address 127.18.45.254:40873)
0806 22:45:27.172856 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:27.172858 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:27.172931 (+ 73us) server_negotiation.cc:407] Connection header received
0806 22:45:27.173016 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:27.173017 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:27.173033 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:27.173093 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:27.173559 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:27.174401 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:27.175254 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:27.175470 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:27.175531 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:27.178481 (+ 2950us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.45.193:60135: BlockingRecv error: recv got EOF from 127.18.45.193:60135 (error 108)
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":98,"threads_started":1}
Aug 06 22:45:27 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.194@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:45:27 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.194@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:45:27.330951 31873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:27.324653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:40873 (local address 127.18.45.194:42163)
0806 22:45:27.325026 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:27.325042 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:27.325131 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:27.325334 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:27.325336 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:27.325592 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:27.325661 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:27.325664 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:27.326463 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:27.326468 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:27.327100 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:27.327102 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:27.327491 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:27.327491 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:27.327559 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:27.328187 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:27.328194 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:27.330865 (+ 2671us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.45.254:40873: Server oryx/127.18.45.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":127,"threads_started":1}
W20260806 22:45:27.331156 31871 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:45:27.324852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:42163 (local address 127.18.45.254:40873)
0806 22:45:27.325082 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:27.325084 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:27.325100 (+ 16us) server_negotiation.cc:407] Connection header received
0806 22:45:27.325226 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:27.325227 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:27.325241 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:27.325325 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:27.325749 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:27.326348 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:27.327192 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:27.327411 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:27.327489 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:27.331072 (+ 3583us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.45.194:42163: BlockingRecv error: recv got EOF from 127.18.45.194:42163 (error 108)
Metrics: {"server-negotiator.queue_time_us":191}
Aug 06 22:45:27 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.195@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:45:27 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.195@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:45:27.504182 31875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:27.498235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:40873 (local address 127.18.45.195:35367)
0806 22:45:27.498523 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:27.498538 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:27.498624 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:27.498801 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:27.498802 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:27.499072 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:27.499143 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:27.499145 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:27.499954 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:27.499955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:27.500570 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:27.500573 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:27.500975 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:27.500976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:27.501042 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:27.501778 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:27.501784 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:27.504103 (+ 2319us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.45.254:40873: Server oryx/127.18.45.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":211,"threads_started":1}
W20260806 22:45:27.504333 31871 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:45:27.498319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.195:35367 (local address 127.18.45.254:40873)
0806 22:45:27.498395 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:27.498397 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:27.498649 (+ 252us) server_negotiation.cc:407] Connection header received
0806 22:45:27.498682 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:27.498683 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:27.498707 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:27.498778 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:27.499240 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:27.499840 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:27.500663 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:27.500873 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:27.500935 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:27.504232 (+ 3297us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.45.195:35367: BlockingRecv error: recv got EOF from 127.18.45.195:35367 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 06 22:45:28 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.193@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:45:28 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.193@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:45:28.184724 31876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:28.179006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:40873 (local address 127.18.45.193:41561)
0806 22:45:28.179230 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:28.179246 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:28.179329 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:28.179507 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:28.179508 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:28.179731 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:28.179807 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:28.179808 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:28.180565 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:28.180566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:28.181224 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:28.181227 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:28.181604 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:28.181605 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:28.181661 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:28.182244 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:28.182249 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:28.184641 (+ 2392us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.45.254:40873: Server oryx/127.18.45.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":98,"threads_started":1}
W20260806 22:45:28.184885 31877 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:45:28.179102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:41561 (local address 127.18.45.254:40873)
0806 22:45:28.179296 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:28.179298 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:28.179308 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:45:28.179399 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:28.179399 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:28.179414 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:28.179485 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:28.179881 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:28.180460 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:28.181312 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:28.181516 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:28.181577 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:28.184755 (+ 3178us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.45.193:41561: BlockingRecv error: recv got EOF from 127.18.45.193:41561 (error 108)
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":102,"threads_started":1}
Aug 06 22:45:28 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.194@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:45:28 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.194@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:45:28.337697 31878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:28.331596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:40873 (local address 127.18.45.194:48947)
0806 22:45:28.331888 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:28.331901 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:28.332011 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:28.332220 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:28.332222 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:28.332478 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:28.332553 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:28.332555 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:28.333325 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:28.333327 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:28.333905 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:28.333906 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:28.334299 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:28.334300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:28.334366 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:28.334925 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:28.334930 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:28.337572 (+ 2642us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.45.254:40873: Server oryx/127.18.45.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":115,"threads_started":1}
W20260806 22:45:28.337890 31877 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:45:28.331707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:48947 (local address 127.18.45.254:40873)
0806 22:45:28.331818 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:28.331822 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:28.331981 (+ 159us) server_negotiation.cc:407] Connection header received
0806 22:45:28.332102 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:28.332103 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:28.332126 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:28.332199 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:28.332661 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:28.333230 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:28.333997 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:28.334219 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:28.334302 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:28.337780 (+ 3478us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.45.194:48947: BlockingRecv error: recv got EOF from 127.18.45.194:48947 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 06 22:45:28 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.195@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:45:28 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.195@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:45:28.511107 31880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:28.504998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:40873 (local address 127.18.45.195:54957)
0806 22:45:28.505264 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:28.505279 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:28.505395 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:28.505612 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:28.505613 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:28.505857 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:28.506028 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:28.506033 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:28.507088 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:28.507089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:28.507685 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:28.507688 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:28.508036 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:28.508036 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:28.508082 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:28.508637 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:28.508642 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:28.510956 (+ 2314us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.45.254:40873: Server oryx/127.18.45.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":143,"threads_started":1}
W20260806 22:45:28.511163 31877 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:45:28.505132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.195:54957 (local address 127.18.45.254:40873)
0806 22:45:28.505251 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:28.505255 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:28.505371 (+ 116us) server_negotiation.cc:407] Connection header received
0806 22:45:28.505497 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:28.505499 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:28.505519 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:28.505599 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:28.506199 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:28.506927 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:28.507766 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:28.507963 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:28.508029 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:28.511086 (+ 3057us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.45.195:54957: BlockingRecv error: recv got EOF from 127.18.45.195:54957 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 06 22:45:29 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.193@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:45:29 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.193@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:45:29.191044 31881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:29.185392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:40873 (local address 127.18.45.193:54125)
0806 22:45:29.185615 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:29.185639 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:29.185728 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:29.185909 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:29.185910 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:29.186150 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:29.186223 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:29.186225 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:29.187043 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:29.187044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:29.187626 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:29.187629 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:29.188018 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:29.188019 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:29.188071 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:29.188713 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:29.188718 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:29.190896 (+ 2178us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.45.254:40873: Server oryx/127.18.45.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":100,"threads_started":1}
W20260806 22:45:29.191231 31882 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:45:29.185461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:54125 (local address 127.18.45.254:40873)
0806 22:45:29.185667 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:29.185669 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:29.185703 (+ 34us) server_negotiation.cc:407] Connection header received
0806 22:45:29.185783 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:29.185783 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:29.185797 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:29.185887 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:29.186324 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:29.186914 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:29.187716 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:29.187939 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:29.188000 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:29.191092 (+ 3092us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.45.193:54125: BlockingRecv error: recv got EOF from 127.18.45.193:54125 (error 108)
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":115,"threads_started":1}
Aug 06 22:45:29 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.194@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:45:29 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.194@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:45:29.344537 31883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:29.338481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:40873 (local address 127.18.45.194:40265)
0806 22:45:29.338702 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:29.338716 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:29.338799 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:29.338988 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:29.338989 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:29.339219 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:29.339330 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:29.339334 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:29.340317 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:29.340319 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:29.341264 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:29.341269 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:29.341627 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:29.341629 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:29.341674 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:29.342258 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:29.342265 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:29.344449 (+ 2184us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.45.254:40873: Server oryx/127.18.45.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":145,"threads_started":1}
W20260806 22:45:29.344653 31882 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:45:29.338567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:40265 (local address 127.18.45.254:40873)
0806 22:45:29.338656 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:29.338659 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:29.338795 (+ 136us) server_negotiation.cc:407] Connection header received
0806 22:45:29.338873 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:29.338873 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:29.338886 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:29.338963 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:29.339444 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:29.340232 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:29.341353 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:29.341553 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:29.341610 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:29.344566 (+ 2956us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.45.194:40265: BlockingRecv error: recv got EOF from 127.18.45.194:40265 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 06 22:45:29 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.195@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:45:29 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.195@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:45:29.524893 31885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:29.511698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:40873 (local address 127.18.45.195:42803)
0806 22:45:29.511887 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:29.511900 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:29.511986 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:29.512147 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:29.512148 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:29.512362 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:29.512449 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:29.512451 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:29.513233 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:29.513234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:29.513857 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:29.513858 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:29.514459 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:29.514460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:29.514529 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:29.522544 (+ 8015us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:29.522552 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:29.524801 (+ 2249us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.45.254:40873: Server oryx/127.18.45.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":108,"threads_started":1}
W20260806 22:45:29.525096 31882 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:45:29.511803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.195:42803 (local address 127.18.45.254:40873)
0806 22:45:29.511918 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:29.511920 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:29.511979 (+ 59us) server_negotiation.cc:407] Connection header received
0806 22:45:29.512055 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:29.512055 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:29.512072 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:29.512120 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:29.512526 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:29.513129 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:29.514110 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:29.514341 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:29.514417 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:29.525008 (+ 10591us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.45.195:42803: BlockingRecv error: recv got EOF from 127.18.45.195:42803 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 06 22:45:30 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.193@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:45:30 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.193@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:45:30.197467 31887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:30.191782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:40873 (local address 127.18.45.193:36157)
0806 22:45:30.192078 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:30.192091 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:30.192174 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:30.192335 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:30.192336 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:30.192584 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:30.192661 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:30.192664 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:30.193497 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:30.193499 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:30.194095 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:30.194097 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:30.194472 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:30.194472 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:30.194516 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:30.195179 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:30.195186 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:30.197328 (+ 2142us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.45.254:40873: Server oryx/127.18.45.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":145,"threads_started":1}
W20260806 22:45:30.197507 31888 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:45:30.191961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:36157 (local address 127.18.45.254:40873)
0806 22:45:30.192100 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:30.192102 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:30.192149 (+ 47us) server_negotiation.cc:407] Connection header received
0806 22:45:30.192224 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:30.192225 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:30.192239 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:30.192306 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:30.192765 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:30.193398 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:30.194182 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:30.194397 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:30.194458 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:30.197425 (+ 2967us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.45.193:36157: BlockingRecv error: recv got EOF from 127.18.45.193:36157 (error 108)
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":70,"threads_started":1}
Aug 06 22:45:30 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.194@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:45:30 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.194@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:45:30.350835 31889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:30.345146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:40873 (local address 127.18.45.194:39851)
0806 22:45:30.345376 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:30.345396 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:30.345504 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:30.345680 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:30.345681 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:30.345937 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:30.346013 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:30.346016 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:30.346904 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:30.346906 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:30.347533 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:30.347535 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:30.347971 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:30.347972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:30.348046 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:30.348663 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:30.348669 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:30.350717 (+ 2048us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.45.254:40873: Server oryx/127.18.45.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":103,"threads_started":1}
W20260806 22:45:30.350891 31888 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:45:30.345247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:39851 (local address 127.18.45.254:40873)
0806 22:45:30.345346 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:30.345348 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:30.345477 (+ 129us) server_negotiation.cc:407] Connection header received
0806 22:45:30.345588 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:30.345589 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:30.345605 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:30.345670 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:30.346107 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:30.346806 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:30.347627 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:30.347852 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:30.347931 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:30.350814 (+ 2883us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.45.194:39851: BlockingRecv error: recv got EOF from 127.18.45.194:39851 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 06 22:45:30 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.195@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:45:30 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.195@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:45:30.531500 31892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:30.525522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:40873 (local address 127.18.45.195:53303)
0806 22:45:30.525738 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:30.525751 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:30.525831 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:30.526024 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:30.526025 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:30.526249 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:30.526323 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:30.526325 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:30.527241 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:30.527242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:30.527821 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:30.527823 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:30.528210 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:30.528211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:30.528255 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:30.528851 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:30.528857 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:30.531344 (+ 2487us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.45.254:40873: Server oryx/127.18.45.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":147,"threads_started":1}
W20260806 22:45:30.531597 31888 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:45:30.525590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.195:53303 (local address 127.18.45.254:40873)
0806 22:45:30.525676 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:30.525681 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:30.525892 (+ 211us) server_negotiation.cc:407] Connection header received
0806 22:45:30.525926 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:30.525927 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:30.525946 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:30.525998 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:30.526435 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:30.527135 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:30.527928 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:30.528131 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:30.528200 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:30.531507 (+ 3307us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.45.195:53303: BlockingRecv error: recv got EOF from 127.18.45.195:53303 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 06 22:45:31 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.193@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:45:31 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.193@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:45:31.204051 31893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:31.198134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:40873 (local address 127.18.45.193:39109)
0806 22:45:31.198315 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:31.198335 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:31.198437 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:31.198642 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:31.198644 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:31.198876 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:31.198954 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:31.198957 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:31.199798 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:31.199799 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:31.200396 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:31.200398 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:31.200775 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:31.200775 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:31.200821 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:31.201370 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:31.201375 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:31.203953 (+ 2578us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.45.254:40873: Server oryx/127.18.45.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":87,"threads_started":1}
W20260806 22:45:31.204238 31894 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:45:31.198257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:39109 (local address 127.18.45.254:40873)
0806 22:45:31.198431 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:31.198433 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:31.198444 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:45:31.198508 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:31.198508 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:31.198523 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:31.198590 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:31.199091 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:31.199712 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:31.200506 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:31.200707 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:31.200760 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:31.204113 (+ 3353us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.45.193:39109: BlockingRecv error: recv got EOF from 127.18.45.193:39109 (error 108)
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":92,"threads_started":1}
Aug 06 22:45:31 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.194@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:45:31 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.194@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:45:31.357527 31896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:31.351491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:40873 (local address 127.18.45.194:44189)
0806 22:45:31.351697 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:31.351710 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:31.351784 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:31.351982 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:31.351984 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:31.352223 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:31.352297 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:31.352299 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:31.353227 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:31.353228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:31.353924 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:31.353927 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:31.354373 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:31.354387 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:31.354445 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:31.355120 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:31.355125 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:31.357431 (+ 2306us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.45.254:40873: Server oryx/127.18.45.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":107,"threads_started":1}
W20260806 22:45:31.357674 31894 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:45:31.351596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:44189 (local address 127.18.45.254:40873)
0806 22:45:31.351726 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:31.351729 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:31.351765 (+ 36us) server_negotiation.cc:407] Connection header received
0806 22:45:31.351865 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:31.351866 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:31.351880 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:31.351963 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:31.352402 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:31.353120 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:31.354041 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:31.354267 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:31.354336 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:31.357553 (+ 3217us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.45.194:44189: BlockingRecv error: recv got EOF from 127.18.45.194:44189 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
Aug 06 22:45:31 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.195@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:45:31 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.195@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:45:31.537767 31897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:31.532154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:40873 (local address 127.18.45.195:38137)
0806 22:45:31.532356 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:31.532370 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:31.532453 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:31.532618 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:31.532619 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:31.532849 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:31.532931 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:31.532933 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:31.533704 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:31.533705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:31.534317 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:31.534319 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:31.534702 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:31.534703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:31.534757 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:31.535404 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:31.535411 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:31.537632 (+ 2221us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.45.254:40873: Server oryx/127.18.45.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":107,"threads_started":1}
W20260806 22:45:31.538079 31894 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:45:31.532283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.195:38137 (local address 127.18.45.254:40873)
0806 22:45:31.532379 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:31.532384 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:31.532448 (+ 64us) server_negotiation.cc:407] Connection header received
0806 22:45:31.532520 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:31.532521 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:31.532538 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:31.532594 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:31.533036 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:31.533619 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:31.534403 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:31.534614 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:31.534672 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:31.537795 (+ 3123us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.45.195:38137: BlockingRecv error: recv got EOF from 127.18.45.195:38137 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 06 22:45:32 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.193@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:45:32 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.193@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:45:32.211175 31898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:32.204796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:40873 (local address 127.18.45.193:39433)
0806 22:45:32.205022 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:32.205038 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:32.205118 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:32.205296 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:32.205298 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:32.205597 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:32.205698 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:32.205702 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:32.206770 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:32.206772 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:32.207488 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:32.207493 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:32.207946 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:32.207948 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:32.208007 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:32.208597 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:32.208603 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:32.211088 (+ 2485us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.45.254:40873: Server oryx/127.18.45.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":147,"threads_started":1}
W20260806 22:45:32.211246 31899 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:45:32.204941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:39433 (local address 127.18.45.254:40873)
0806 22:45:32.205124 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:32.205128 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:32.205167 (+ 39us) server_negotiation.cc:407] Connection header received
0806 22:45:32.205195 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:32.205196 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:32.205208 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:32.205273 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:32.205814 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:32.206671 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:32.207602 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:32.207824 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:32.207940 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:32.211177 (+ 3237us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.45.193:39433: BlockingRecv error: recv got EOF from 127.18.45.193:39433 (error 108)
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":116,"threads_started":1}
Aug 06 22:45:32 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.194@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:45:32 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.194@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:45:32.364331 31900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:32.358117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:40873 (local address 127.18.45.194:52343)
0806 22:45:32.358409 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:32.358428 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:32.358537 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:32.358735 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:32.358736 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:32.359071 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:32.359142 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:32.359145 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:32.359949 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:32.359950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:32.360533 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:32.360535 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:32.360933 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:32.360934 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:32.361004 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:32.361610 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:32.361615 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:32.364245 (+ 2630us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.45.254:40873: Server oryx/127.18.45.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":163,"threads_started":1}
W20260806 22:45:32.364539 31899 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:45:32.358265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:52343 (local address 127.18.45.254:40873)
0806 22:45:32.358385 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:32.358388 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:32.358509 (+ 121us) server_negotiation.cc:407] Connection header received
0806 22:45:32.358628 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:32.358629 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:32.358651 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:32.358727 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:32.359235 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:32.359848 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:32.360628 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:32.360840 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:32.360923 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:32.364447 (+ 3524us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.45.194:52343: BlockingRecv error: recv got EOF from 127.18.45.194:52343 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 06 22:45:32 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.195@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:45:32 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.195@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:45:32.544157 31901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:32.538458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:40873 (local address 127.18.45.195:36725)
0806 22:45:32.538662 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:32.538675 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:32.538745 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:32.538890 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:32.538890 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:32.539147 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:32.539217 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:32.539219 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:32.540198 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:32.540199 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:32.540775 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:32.540776 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:32.541157 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:32.541158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:32.541207 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:32.541831 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:32.541836 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:32.544076 (+ 2240us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.45.254:40873: Server oryx/127.18.45.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":128,"threads_started":1}
W20260806 22:45:32.544323 31899 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:45:32.538604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.195:36725 (local address 127.18.45.254:40873)
0806 22:45:32.538677 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:32.538679 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:32.538740 (+ 61us) server_negotiation.cc:407] Connection header received
0806 22:45:32.538793 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:32.538793 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:32.538810 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:32.538867 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:32.539301 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:32.540015 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:32.540864 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:32.541077 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:32.541137 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:32.544225 (+ 3088us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.45.195:36725: BlockingRecv error: recv got EOF from 127.18.45.195:36725 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 06 22:45:33 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.193@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:45:33 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.193@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:45:33.217657 31902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:33.212015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:40873 (local address 127.18.45.193:54173)
0806 22:45:33.212241 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:33.212254 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:33.212332 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:33.212479 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:33.212481 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:33.212723 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:33.212803 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:33.212805 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:33.213585 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:33.213586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:33.214183 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:33.214185 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:33.214583 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:33.214584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:33.214629 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:33.215324 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:33.215330 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:33.217523 (+ 2193us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.45.254:40873: Server oryx/127.18.45.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":146,"threads_started":1}
W20260806 22:45:33.217705 31906 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:45:33.212120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:54173 (local address 127.18.45.254:40873)
0806 22:45:33.212273 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:33.212275 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:33.212325 (+ 50us) server_negotiation.cc:407] Connection header received
0806 22:45:33.212376 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:33.212377 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:33.212391 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:33.212462 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:33.212900 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:33.213508 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:33.214266 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:33.214506 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:33.214571 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:33.217615 (+ 3044us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.45.193:54173: BlockingRecv error: recv got EOF from 127.18.45.193:54173 (error 108)
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":77,"threads_started":1}
Aug 06 22:45:33 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.194@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:45:33 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.194@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:45:33.370463 31907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:33.364867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:40873 (local address 127.18.45.194:47411)
0806 22:45:33.365031 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:33.365044 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:33.365174 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:33.365351 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:33.365352 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:33.365583 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:33.365646 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:33.365647 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:33.366400 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:33.366401 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:33.367183 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:33.367185 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:33.367561 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:33.367562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:33.367612 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:33.368208 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:33.368214 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:33.370376 (+ 2162us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.45.254:40873: Server oryx/127.18.45.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":79,"threads_started":1}
W20260806 22:45:33.370604 31906 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:45:33.364954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:47411 (local address 127.18.45.254:40873)
0806 22:45:33.365033 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:33.365036 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:33.365154 (+ 118us) server_negotiation.cc:407] Connection header received
0806 22:45:33.365257 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:33.365258 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:33.365273 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:33.365339 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:33.365751 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:33.366305 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:33.367261 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:33.367485 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:33.367553 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:33.370494 (+ 2941us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.45.194:47411: BlockingRecv error: recv got EOF from 127.18.45.194:47411 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 06 22:45:33 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.195@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:45:33 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.195@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:45:33.550853 31908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:33.544768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:40873 (local address 127.18.45.195:48345)
0806 22:45:33.545021 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:33.545035 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:33.545160 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:33.545403 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:33.545404 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:33.545666 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:33.545737 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:33.545740 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:33.546575 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:33.546576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:33.547283 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:33.547287 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:33.547712 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:33.547713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:33.547790 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:33.548397 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:33.548403 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:33.550710 (+ 2307us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.45.254:40873: Server oryx/127.18.45.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":98,"threads_started":1}
W20260806 22:45:33.550905 31906 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:45:33.544849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.195:48345 (local address 127.18.45.254:40873)
0806 22:45:33.544959 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:33.544962 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:33.545129 (+ 167us) server_negotiation.cc:407] Connection header received
0806 22:45:33.545268 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:33.545269 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:33.545296 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:33.545390 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:33.545858 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:33.546458 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:33.547393 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:33.547611 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:33.547709 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:33.550825 (+ 3116us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.45.195:48345: BlockingRecv error: recv got EOF from 127.18.45.195:48345 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 06 22:45:34 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.193@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:45:34 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.193@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:45:34.224670 31909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:34.218479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:40873 (local address 127.18.45.193:52395)
0806 22:45:34.218727 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:34.218740 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:34.218812 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:34.218981 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:34.218983 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:34.219248 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:34.219315 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:34.219318 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:34.220141 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:34.220143 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:34.220929 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:34.220932 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:34.221310 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:34.221311 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:34.221358 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:34.222000 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:34.222005 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:34.224538 (+ 2533us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.45.254:40873: Server oryx/127.18.45.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":146,"threads_started":1}
W20260806 22:45:34.224782 31911 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:45:34.218616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:52395 (local address 127.18.45.254:40873)
0806 22:45:34.218812 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:34.218814 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:34.218846 (+ 32us) server_negotiation.cc:407] Connection header received
0806 22:45:34.218874 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:34.218875 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:34.218888 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:34.218946 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:34.219401 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:34.220044 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:34.221023 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:34.221239 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:34.221318 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:34.224685 (+ 3367us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.45.193:52395: BlockingRecv error: recv got EOF from 127.18.45.193:52395 (error 108)
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":128,"threads_started":1}
Aug 06 22:45:34 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.194@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:45:34 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.194@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:45:34.376713 31915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:34.371113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:40873 (local address 127.18.45.194:54409)
0806 22:45:34.371338 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:34.371352 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:34.371455 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:34.371598 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:34.371599 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:34.371845 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:34.371924 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:34.371926 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:34.372672 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:34.372673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:34.373398 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:34.373402 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:34.373817 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:34.373818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:34.373868 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:34.374471 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:34.374476 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:34.376617 (+ 2141us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.45.254:40873: Server oryx/127.18.45.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":102,"threads_started":1}
W20260806 22:45:34.376912 31911 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:45:34.371263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:54409 (local address 127.18.45.254:40873)
0806 22:45:34.371341 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:34.371343 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:34.371412 (+ 69us) server_negotiation.cc:407] Connection header received
0806 22:45:34.371511 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:34.371512 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:34.371527 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:34.371580 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:34.372006 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:34.372586 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:34.373507 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:34.373743 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:34.373809 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:34.376818 (+ 3009us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.45.194:54409: BlockingRecv error: recv got EOF from 127.18.45.194:54409 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 06 22:45:34 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.195@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:45:34 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.195@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:45:34.562650 31916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:34.551554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:40873 (local address 127.18.45.195:47345)
0806 22:45:34.551800 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:34.551815 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:34.551959 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:34.552145 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:34.552147 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:34.552436 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:34.552598 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:34.552602 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:34.553462 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:34.553464 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:34.554078 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:34.554080 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:34.554475 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:34.554477 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:34.554568 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:34.559943 (+ 5375us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:34.559951 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:34.562477 (+ 2526us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.45.254:40873: Server oryx/127.18.45.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":121,"threads_started":1}
W20260806 22:45:34.562695 31911 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:45:34.551695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.195:47345 (local address 127.18.45.254:40873)
0806 22:45:34.551806 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:34.551809 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:34.551911 (+ 102us) server_negotiation.cc:407] Connection header received
0806 22:45:34.552051 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:34.552053 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:34.552073 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:34.552137 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:34.552697 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:34.553373 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:34.554170 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:34.554381 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:34.554472 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:34.562615 (+ 8143us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.45.195:47345: BlockingRecv error: recv got EOF from 127.18.45.195:47345 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 06 22:45:35 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.193@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:45:35 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.193@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:45:35.232226 31917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:35.225401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:40873 (local address 127.18.45.193:51559)
0806 22:45:35.225640 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:35.225654 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:35.225735 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:35.226126 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:35.226128 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:35.226381 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:35.226686 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:35.226692 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:35.227601 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:35.227602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:35.228273 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:35.228276 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:35.228890 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:35.228892 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:35.228975 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:35.229615 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:35.229621 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:35.232144 (+ 2523us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.45.254:40873: Server oryx/127.18.45.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":132,"threads_started":1}
W20260806 22:45:35.232491 31919 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:45:35.225534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:51559 (local address 127.18.45.254:40873)
0806 22:45:35.225905 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:35.225907 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:35.225920 (+ 13us) server_negotiation.cc:407] Connection header received
0806 22:45:35.225964 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:35.225965 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:35.225981 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:35.226073 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:35.226811 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:35.227481 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:35.228410 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:35.228746 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:35.228851 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:35.232370 (+ 3519us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.45.193:51559: BlockingRecv error: recv got EOF from 127.18.45.193:51559 (error 108)
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":123,"threads_started":1}
Aug 06 22:45:35 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.194@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:45:35 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.194@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:45:35.383495 31921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:35.377353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:40873 (local address 127.18.45.194:33711)
0806 22:45:35.377606 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:35.377623 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:35.377719 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:35.377979 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:35.377981 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:35.378245 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:35.378319 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:35.378322 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:35.379230 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:35.379231 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:35.379881 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:35.379883 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:35.380323 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:35.380325 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:35.380394 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:35.381030 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:35.381036 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:35.383388 (+ 2352us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.45.254:40873: Server oryx/127.18.45.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":123,"threads_started":1}
W20260806 22:45:35.383582 31919 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:45:35.377514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:33711 (local address 127.18.45.254:40873)
0806 22:45:35.377607 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:35.377610 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:35.377689 (+ 79us) server_negotiation.cc:407] Connection header received
0806 22:45:35.377815 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:35.377816 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:35.377840 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:35.377917 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:35.378425 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:35.379127 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:35.379989 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:35.380232 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:35.380332 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:35.383520 (+ 3188us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.45.194:33711: BlockingRecv error: recv got EOF from 127.18.45.194:33711 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 06 22:45:35 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.195@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:45:35 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.195@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:45:35.570289 31922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:35.563418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:40873 (local address 127.18.45.195:52201)
0806 22:45:35.563648 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:35.563662 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:35.563775 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:35.564058 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:35.564060 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:35.564416 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:35.564522 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:35.564526 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:35.565451 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:35.565452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:35.566062 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:35.566064 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:35.566538 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:35.566540 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:35.566614 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:35.567353 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:35.567359 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:35.570210 (+ 2851us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.45.254:40873: Server oryx/127.18.45.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":153,"threads_started":1}
W20260806 22:45:35.570482 31919 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:45:35.563493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.195:52201 (local address 127.18.45.254:40873)
0806 22:45:35.563585 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:35.563588 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:35.563756 (+ 168us) server_negotiation.cc:407] Connection header received
0806 22:45:35.563860 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:35.563861 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:35.563905 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:35.563989 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:35.564619 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:35.565296 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:35.566151 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:35.566451 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:35.566554 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:35.570328 (+ 3774us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.45.195:52201: BlockingRecv error: recv got EOF from 127.18.45.195:52201 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 06 22:45:36 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.193@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:45:36 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.193@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:45:36.238845 31924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:36.232996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:40873 (local address 127.18.45.193:48691)
0806 22:45:36.233331 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:36.233344 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:36.233421 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:36.233587 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:36.233588 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:36.233821 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:36.233893 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:36.233896 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:36.234667 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:36.234668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:36.235272 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:36.235273 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:36.235649 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:36.235650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:36.235690 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:36.236368 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:36.236373 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:36.238771 (+ 2398us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.45.254:40873: Server oryx/127.18.45.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":224,"threads_started":1}
W20260806 22:45:36.239006 31925 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:45:36.233231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:48691 (local address 127.18.45.254:40873)
0806 22:45:36.233398 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:36.233400 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:36.233412 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:45:36.233480 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:36.233481 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:36.233493 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:36.233561 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:36.233997 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:36.234576 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:36.235356 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:36.235580 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:36.235639 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:36.238892 (+ 3253us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.45.193:48691: BlockingRecv error: recv got EOF from 127.18.45.193:48691 (error 108)
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":90,"threads_started":1}
Aug 06 22:45:36 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.194@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:45:36 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.194@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:45:36.389537 31926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:36.384135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:40873 (local address 127.18.45.194:50873)
0806 22:45:36.384361 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:36.384381 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:36.384450 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:36.384604 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:36.384605 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:36.384831 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:36.384899 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:36.384901 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:36.385694 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:36.385694 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:36.386312 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:36.386314 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:36.386683 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:36.386684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:36.386729 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:36.387389 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:36.387395 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:36.389459 (+ 2064us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.45.254:40873: Server oryx/127.18.45.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":150,"threads_started":1}
W20260806 22:45:36.389703 31925 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:45:36.384265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:50873 (local address 127.18.45.254:40873)
0806 22:45:36.384361 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:36.384363 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:36.384430 (+ 67us) server_negotiation.cc:407] Connection header received
0806 22:45:36.384505 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:36.384506 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:36.384524 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:36.384580 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:36.385001 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:36.385598 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:36.386393 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:36.386608 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:36.386667 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:36.389608 (+ 2941us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.45.194:50873: BlockingRecv error: recv got EOF from 127.18.45.194:50873 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 06 22:45:36 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.195@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:45:36 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.195@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:45:36.577517 31927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:36.570888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:40873 (local address 127.18.45.195:40053)
0806 22:45:36.571123 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:36.571141 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:36.571231 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:36.571410 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:36.571412 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:36.571648 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:36.571716 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:36.571719 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:36.572587 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:36.572588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:36.573482 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:36.573487 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:36.574078 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:36.574079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:36.574155 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:36.574872 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:36.574879 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:36.577388 (+ 2509us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.45.254:40873: Server oryx/127.18.45.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":144,"threads_started":1}
W20260806 22:45:36.577591 31925 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:45:36.570983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.195:40053 (local address 127.18.45.254:40873)
0806 22:45:36.571142 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:36.571144 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:36.571220 (+ 76us) server_negotiation.cc:407] Connection header received
0806 22:45:36.571283 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:36.571284 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:36.571306 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:36.571362 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:36.571790 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:36.572484 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:36.573580 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:36.573994 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:36.574076 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:36.577513 (+ 3437us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.45.195:40053: BlockingRecv error: recv got EOF from 127.18.45.195:40053 (error 108)
Metrics: {"server-negotiator.queue_time_us":118}
Aug 06 22:45:37 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.193@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:45:37 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.193@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:45:37.245416 31928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:37.239576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:40873 (local address 127.18.45.193:45743)
0806 22:45:37.239765 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:37.239779 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:37.239848 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:37.240085 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:37.240085 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:37.240313 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:37.240392 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:37.240394 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:37.241149 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:37.241150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:37.241754 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:37.241756 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:37.242153 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:37.242154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:37.242203 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:37.242990 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:37.243048 (+ 58us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:37.245267 (+ 2219us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.45.254:40873: Server oryx/127.18.45.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":114,"threads_started":1}
W20260806 22:45:37.245463 31929 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:45:37.239793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:45743 (local address 127.18.45.254:40873)
0806 22:45:37.239943 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:37.239945 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:37.239952 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:45:37.239977 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:37.239978 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:37.239989 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:37.240061 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:37.240464 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:37.241048 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:37.241836 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:37.242078 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:37.242136 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:37.245372 (+ 3236us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.45.193:45743: BlockingRecv error: recv got EOF from 127.18.45.193:45743 (error 108)
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":85,"threads_started":1}
Aug 06 22:45:37 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.194@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
Aug 06 22:45:37 dist-test-slave-h3ft krb5kdc[30386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.45.194@KRBTEST.COM for oryx/127.18.45.254@KRBTEST.COM, Server not found in Kerberos database
I20260806 22:45:37.396548 31931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:37.390363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:40873 (local address 127.18.45.194:40205)
0806 22:45:37.390666 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:37.390678 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:37.390757 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:37.390931 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:37.390932 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:37.391193 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:37.391264 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:37.391266 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:37.392081 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:37.392082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:37.392734 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:37.392737 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:37.393147 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:37.393148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:37.393205 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:37.394160 (+ 955us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:37.394166 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:37.396454 (+ 2288us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.45.254:40873: Server oryx/127.18.45.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":218,"threads_started":1}
W20260806 22:45:37.396740 31929 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:45:37.390465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:40205 (local address 127.18.45.254:40873)
0806 22:45:37.390564 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:37.390566 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:37.390755 (+ 189us) server_negotiation.cc:407] Connection header received
0806 22:45:37.390831 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:37.390832 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:37.390850 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:37.390900 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:37.391367 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:37.391957 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:37.392821 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:37.393058 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:37.393136 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:37.396634 (+ 3498us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.45.194:40205: BlockingRecv error: recv got EOF from 127.18.45.194:40205 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 22:45:37.459581 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 31234
I20260806 22:45:37.488699 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 31462
I20260806 22:45:37.500952 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 31654
I20260806 22:45:37.518497 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 31127
2026-08-06T22:45:37Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18188 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-06T22:45:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:45:37Z Disabled control of system clock
I20260806 22:45:37.562593 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056275803658-18615-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056275803658-18615-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056275803658-18615-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:41475
--webserver_interface=127.18.45.254
--webserver_port=0
--builtin_ntp_servers=127.18.45.212:38745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.45.254:41475
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 22:45:37.663686 31935 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:45:37.663990 31935 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:45:37.664013 31935 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:45:37.665426 31935 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:45:37.665486 31935 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:45:37.665499 31935 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:45:37.665513 31935 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:45:37.665532 31935 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:45:37.667204 31935 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:38745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056275803658-18615-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056275803658-18615-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.45.254:41475
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:41475
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.45.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31935
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056275803658-18615-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:45:37.667517 31935 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:45:37.667865 31935 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:45:37.674657 31942 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:37.674675 31943 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:37.674731 31935 server_base.cc:1034] running on GCE node
W20260806 22:45:37.674690 31945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:37.675282 31935 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:45:37.675685 31935 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:45:37.676892 31935 hybrid_clock.cc:648] HybridClock initialized: now 1786056337676881 us; error 51 us; skew 500 ppm
I20260806 22:45:37.678651 31935 webserver.cc:460] Webserver started at http://127.18.45.254:45363/ using document root <none> and password file <none>
I20260806 22:45:37.678958 31935 fs_manager.cc:359] Metadata directory not provided
I20260806 22:45:37.679064 31935 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:45:37.679248 31935 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:45:37.680572 31935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056275803658-18615-0/minicluster-data/master-0/data/instance:
uuid: "86cc35198aa249c088adfefa444bc857"
format_stamp: "Formatted at 2026-08-06 22:45:37 on dist-test-slave-h3ft"
server_key: "5495989e2c7e5d185ac9249751ff1979"
server_key_iv: "3a09a16b7efc61c75c07a9fd43da71a1"
server_key_version: "encryptionkey@0"
I20260806 22:45:37.681020 31935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056275803658-18615-0/minicluster-data/master-0/wal/instance:
uuid: "86cc35198aa249c088adfefa444bc857"
format_stamp: "Formatted at 2026-08-06 22:45:37 on dist-test-slave-h3ft"
server_key: "5495989e2c7e5d185ac9249751ff1979"
server_key_iv: "3a09a16b7efc61c75c07a9fd43da71a1"
server_key_version: "encryptionkey@0"
I20260806 22:45:37.682545 31935 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:45:37.683506 31953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:37.683701 31935 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:45:37.683823 31935 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056275803658-18615-0/minicluster-data/master-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056275803658-18615-0/minicluster-data/master-0/wal
uuid: "86cc35198aa249c088adfefa444bc857"
format_stamp: "Formatted at 2026-08-06 22:45:37 on dist-test-slave-h3ft"
server_key: "5495989e2c7e5d185ac9249751ff1979"
server_key_iv: "3a09a16b7efc61c75c07a9fd43da71a1"
server_key_version: "encryptionkey@0"
I20260806 22:45:37.683912 31935 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056275803658-18615-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056275803658-18615-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056275803658-18615-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:45:37.713477 31935 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:45:37.713966 31935 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:45:37.714166 31935 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:45:37.719918 31935 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.254:41475
I20260806 22:45:37.719959 32025 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.254:41475 every 8 connection(s)
I20260806 22:45:37.720448 31935 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
I20260806 22:45:37.721334 32026 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:45:37.724639 32026 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86cc35198aa249c088adfefa444bc857: Bootstrap starting.
I20260806 22:45:37.725394 32026 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 86cc35198aa249c088adfefa444bc857: Neither blocks nor log segments found. Creating new log.
I20260806 22:45:37.725761 32026 log.cc:824] T 00000000000000000000000000000000 P 86cc35198aa249c088adfefa444bc857: Log is configured to *not* fsync() on all Append() calls
I20260806 22:45:37.726603 32026 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86cc35198aa249c088adfefa444bc857: No bootstrap required, opened a new log
I20260806 22:45:37.728533 32026 raft_consensus.cc:348] T 00000000000000000000000000000000 P 86cc35198aa249c088adfefa444bc857 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86cc35198aa249c088adfefa444bc857" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 41475 } }
I20260806 22:45:37.728704 32026 raft_consensus.cc:374] T 00000000000000000000000000000000 P 86cc35198aa249c088adfefa444bc857 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:45:37.728730 32026 raft_consensus.cc:729] T 00000000000000000000000000000000 P 86cc35198aa249c088adfefa444bc857 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86cc35198aa249c088adfefa444bc857, State: Initialized, Role: FOLLOWER
I20260806 22:45:37.728921 32026 consensus_queue.cc:260] T 00000000000000000000000000000000 P 86cc35198aa249c088adfefa444bc857 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86cc35198aa249c088adfefa444bc857" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 41475 } }
I20260806 22:45:37.729017 32026 raft_consensus.cc:388] T 00000000000000000000000000000000 P 86cc35198aa249c088adfefa444bc857 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:45:37.729072 32026 raft_consensus.cc:482] T 00000000000000000000000000000000 P 86cc35198aa249c088adfefa444bc857 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:45:37.729128 32026 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 86cc35198aa249c088adfefa444bc857 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:45:37.729403 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 31935
I20260806 22:45:37.729535 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056275803658-18615-0/minicluster-data/master-0/wal/instance
I20260806 22:45:37.729791 18615 external_mini_cluster.cc:1428] Setting key 7ebfb2b40654773270e30ebd7bd53353
I20260806 22:45:37.730087 32026 raft_consensus.cc:504] T 00000000000000000000000000000000 P 86cc35198aa249c088adfefa444bc857 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86cc35198aa249c088adfefa444bc857" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 41475 } }
I20260806 22:45:37.730250 32026 leader_election.cc:302] T 00000000000000000000000000000000 P 86cc35198aa249c088adfefa444bc857 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 86cc35198aa249c088adfefa444bc857; no voters:
I20260806 22:45:37.730532 32026 leader_election.cc:288] T 00000000000000000000000000000000 P 86cc35198aa249c088adfefa444bc857 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:45:37.730561 32029 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 86cc35198aa249c088adfefa444bc857 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:45:37.730880 32029 raft_consensus.cc:686] T 00000000000000000000000000000000 P 86cc35198aa249c088adfefa444bc857 [term 1 LEADER]: Becoming Leader. State: Replica: 86cc35198aa249c088adfefa444bc857, State: Running, Role: LEADER
I20260806 22:45:37.731045 32026 sys_catalog.cc:564] T 00000000000000000000000000000000 P 86cc35198aa249c088adfefa444bc857 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:45:37.731068 32029 consensus_queue.cc:237] T 00000000000000000000000000000000 P 86cc35198aa249c088adfefa444bc857 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86cc35198aa249c088adfefa444bc857" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 41475 } }
I20260806 22:45:37.731613 32034 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86cc35198aa249c088adfefa444bc857 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 86cc35198aa249c088adfefa444bc857. Latest consensus state: current_term: 1 leader_uuid: "86cc35198aa249c088adfefa444bc857" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86cc35198aa249c088adfefa444bc857" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 41475 } } }
I20260806 22:45:37.731720 32034 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86cc35198aa249c088adfefa444bc857 [sys.catalog]: This master's current role is: LEADER
I20260806 22:45:37.731805 32033 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86cc35198aa249c088adfefa444bc857 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "86cc35198aa249c088adfefa444bc857" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86cc35198aa249c088adfefa444bc857" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 41475 } } }
I20260806 22:45:37.731895 32033 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86cc35198aa249c088adfefa444bc857 [sys.catalog]: This master's current role is: LEADER
I20260806 22:45:37.732954 32044 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:45:37.733762 32044 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:45:37.735991 32044 catalog_manager.cc:1347] Generated new cluster ID: 107990f60e1341a0bd3a089fa0c91ee4
I20260806 22:45:37.736063 32044 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:45:37.737970 32037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:37.730721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47656 (local address 127.18.45.254:41475)
0806 22:45:37.731159 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:37.731170 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:37.731208 (+ 38us) server_negotiation.cc:407] Connection header received
0806 22:45:37.731344 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:37.731350 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:37.731414 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:37.731936 (+ 522us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:37.732130 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:37.733659 (+ 1529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:37.734970 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:37.735667 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:37.736772 (+ 1105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:37.736951 (+ 179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:37.736955 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:37.736955 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:45:37.737199 (+ 244us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:37.737240 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:45:37.737255 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:37.737580 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:37.737613 (+ 33us) server_negotiation.cc:299] Negotiation successful
0806 22:45:37.737708 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":136,"threads_started":1}
I20260806 22:45:37.773104 32044 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:45:37.773948 32044 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:45:37.785449 32044 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 86cc35198aa249c088adfefa444bc857: Generated new TSK 0
I20260806 22:45:37.785775 32044 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:45:37.794948 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.193:0
--local_ip_for_outbound_sockets=127.18.45.193
--webserver_interface=127.18.45.193
--webserver_port=0
--tserver_master_addrs=127.18.45.254:41475
--builtin_ntp_servers=127.18.45.212:38745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260806 22:45:37.895087 32059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:45:37.895386 32059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:45:37.895427 32059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:45:37.896727 32059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:45:37.896795 32059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:45:37.896838 32059 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.193
I20260806 22:45:37.898320 32059 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:38745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.45.193
--webserver_port=0
--tserver_master_addrs=127.18.45.254:41475
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32059
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.193
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:45:37.898658 32059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:45:37.899106 32059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:45:37.905073 32066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:37.905073 32065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:37.905073 32068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:37.905478 32059 server_base.cc:1034] running on GCE node
I20260806 22:45:37.905761 32059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:45:37.906220 32059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:45:37.907397 32059 hybrid_clock.cc:648] HybridClock initialized: now 1786056337907375 us; error 43 us; skew 500 ppm
I20260806 22:45:37.909102 32059 webserver.cc:460] Webserver started at http://127.18.45.193:41869/ using document root <none> and password file <none>
I20260806 22:45:37.909399 32059 fs_manager.cc:359] Metadata directory not provided
I20260806 22:45:37.909483 32059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:45:37.909648 32059 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:45:37.910970 32059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-0/data/instance:
uuid: "2a97d56606cd45b0bf1bb61fda23954a"
format_stamp: "Formatted at 2026-08-06 22:45:37 on dist-test-slave-h3ft"
server_key: "460fc04ba482998dabff418fa6301b44"
server_key_iv: "9011683de8c95eb9299b19a90c83f646"
server_key_version: "encryptionkey@0"
I20260806 22:45:37.911463 32059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-0/wal/instance:
uuid: "2a97d56606cd45b0bf1bb61fda23954a"
format_stamp: "Formatted at 2026-08-06 22:45:37 on dist-test-slave-h3ft"
server_key: "460fc04ba482998dabff418fa6301b44"
server_key_iv: "9011683de8c95eb9299b19a90c83f646"
server_key_version: "encryptionkey@0"
I20260806 22:45:37.913048 32059 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:45:37.913938 32077 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:37.914197 32059 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:45:37.914359 32059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-0/wal
uuid: "2a97d56606cd45b0bf1bb61fda23954a"
format_stamp: "Formatted at 2026-08-06 22:45:37 on dist-test-slave-h3ft"
server_key: "460fc04ba482998dabff418fa6301b44"
server_key_iv: "9011683de8c95eb9299b19a90c83f646"
server_key_version: "encryptionkey@0"
I20260806 22:45:37.914466 32059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:45:37.939973 32059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:45:37.940483 32059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:45:37.940707 32059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:45:37.941046 32059 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:45:37.941562 32059 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:45:37.941648 32059 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:37.941705 32059 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:45:37.941761 32059 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:37.950577 32059 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.193:42297
I20260806 22:45:37.950640 32256 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.193:42297 every 8 connection(s)
I20260806 22:45:37.951243 32059 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
I20260806 22:45:37.953290 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 32059
I20260806 22:45:37.953393 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-0/wal/instance
I20260806 22:45:37.953636 18615 external_mini_cluster.cc:1428] Setting key 6c25ea618ea8b3a781d56ba58c1a316e
I20260806 22:45:37.954994 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.194:0
--local_ip_for_outbound_sockets=127.18.45.194
--webserver_interface=127.18.45.194
--webserver_port=0
--tserver_master_addrs=127.18.45.254:41475
--builtin_ntp_servers=127.18.45.212:38745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 22:45:37.958940 32037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:37.952232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:48329 (local address 127.18.45.254:41475)
0806 22:45:37.952338 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:37.952341 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:37.952632 (+ 291us) server_negotiation.cc:407] Connection header received
0806 22:45:37.953494 (+ 862us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:37.953496 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:37.953518 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:37.953623 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:37.954981 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:37.955767 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:37.957095 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:37.957354 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:37.957456 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:37.958560 (+ 1104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:37.958561 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:37.958562 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:45:37.958568 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:37.958588 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:45:37.958594 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:37.958667 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:37.958795 (+ 128us) server_negotiation.cc:299] Negotiation successful
0806 22:45:37.958861 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 22:45:37.959339 32261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:37.952191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:41475 (local address 127.18.45.193:48329)
0806 22:45:37.952443 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:37.952491 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:37.953276 (+ 785us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:37.953746 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:37.953752 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:37.953767 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:37.954803 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:37.954811 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:37.955882 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:37.955883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:37.956981 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:37.956984 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:37.957467 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:37.957468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:37.957634 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:37.958366 (+ 732us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:45:37.958367 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:37.958411 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:45:37.958442 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:45:37.958669 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:45:37.958677 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:45:37.958681 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:45:37.958774 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 22:45:37.958919 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":85,"threads_started":1}
I20260806 22:45:37.960069 32258 heartbeater.cc:347] Connected to a master server at 127.18.45.254:41475
I20260806 22:45:37.960201 32258 heartbeater.cc:464] Registering TS with master...
I20260806 22:45:37.960426 32258 heartbeater.cc:511] Master 127.18.45.254:41475 requested a full tablet report, sending...
I20260806 22:45:37.961153 31978 ts_manager.cc:194] Registered new tserver with Master: 2a97d56606cd45b0bf1bb61fda23954a (127.18.45.193:42297)
I20260806 22:45:37.961853 31978 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.45.193:48329
W20260806 22:45:38.056680 32264 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:45:38.057003 32264 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:45:38.057045 32264 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:45:38.058338 32264 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:45:38.058419 32264 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:45:38.058465 32264 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.194
I20260806 22:45:38.060046 32264 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:38745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.45.194
--webserver_port=0
--tserver_master_addrs=127.18.45.254:41475
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32264
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.194
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:45:38.060389 32264 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:45:38.060798 32264 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:45:38.067242 32273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:38.067242 32272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:38.067274 32276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:38.067615 32264 server_base.cc:1034] running on GCE node
I20260806 22:45:38.067991 32264 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:45:38.068336 32264 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:45:38.069521 32264 hybrid_clock.cc:648] HybridClock initialized: now 1786056338069511 us; error 44 us; skew 500 ppm
I20260806 22:45:38.071426 32264 webserver.cc:460] Webserver started at http://127.18.45.194:42083/ using document root <none> and password file <none>
I20260806 22:45:38.071781 32264 fs_manager.cc:359] Metadata directory not provided
I20260806 22:45:38.071842 32264 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:45:38.072021 32264 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:45:38.073486 32264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-1/data/instance:
uuid: "4c11993958054533b5ad8895935b3947"
format_stamp: "Formatted at 2026-08-06 22:45:38 on dist-test-slave-h3ft"
server_key: "4f9059f379956bd1f8bada2fc37325dd"
server_key_iv: "3db6b96fab3ae57c0dfd610fb3293730"
server_key_version: "encryptionkey@0"
I20260806 22:45:38.073951 32264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-1/wal/instance:
uuid: "4c11993958054533b5ad8895935b3947"
format_stamp: "Formatted at 2026-08-06 22:45:38 on dist-test-slave-h3ft"
server_key: "4f9059f379956bd1f8bada2fc37325dd"
server_key_iv: "3db6b96fab3ae57c0dfd610fb3293730"
server_key_version: "encryptionkey@0"
I20260806 22:45:38.075760 32264 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260806 22:45:38.076799 32287 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:38.077023 32264 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:45:38.077157 32264 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-1/wal
uuid: "4c11993958054533b5ad8895935b3947"
format_stamp: "Formatted at 2026-08-06 22:45:38 on dist-test-slave-h3ft"
server_key: "4f9059f379956bd1f8bada2fc37325dd"
server_key_iv: "3db6b96fab3ae57c0dfd610fb3293730"
server_key_version: "encryptionkey@0"
I20260806 22:45:38.077252 32264 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:45:38.088871 32264 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:45:38.089321 32264 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:45:38.089522 32264 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:45:38.089818 32264 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:45:38.090236 32264 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:45:38.090299 32264 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:38.090349 32264 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:45:38.090391 32264 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:38.098511 32264 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.194:34541
I20260806 22:45:38.098570 32464 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.194:34541 every 8 connection(s)
I20260806 22:45:38.099100 32264 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
I20260806 22:45:38.103551 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 32264
I20260806 22:45:38.103672 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-1/wal/instance
I20260806 22:45:38.103936 18615 external_mini_cluster.cc:1428] Setting key 65ba73d953bf41fbd290f005e9590ff7
I20260806 22:45:38.105394 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.195:0
--local_ip_for_outbound_sockets=127.18.45.195
--webserver_interface=127.18.45.195
--webserver_port=0
--tserver_master_addrs=127.18.45.254:41475
--builtin_ntp_servers=127.18.45.212:38745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 22:45:38.106492 32037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:38.100169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:35033 (local address 127.18.45.254:41475)
0806 22:45:38.100286 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:38.100289 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:38.100513 (+ 224us) server_negotiation.cc:407] Connection header received
0806 22:45:38.101334 (+ 821us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:38.101337 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:38.101355 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:38.101437 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:38.102412 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:38.103069 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:38.104513 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:38.104768 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:38.104845 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:38.106161 (+ 1316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:38.106163 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:38.106163 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:45:38.106168 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:38.106189 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:45:38.106195 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:38.106235 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:38.106370 (+ 135us) server_negotiation.cc:299] Negotiation successful
0806 22:45:38.106404 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 22:45:38.106907 32469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:38.100089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:41475 (local address 127.18.45.194:35033)
0806 22:45:38.100335 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:38.100380 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:38.101086 (+ 706us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:38.101570 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:38.101575 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:38.101588 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:38.102276 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:38.102282 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:38.103194 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:38.103195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:38.104407 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:38.104412 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:38.104952 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:38.104953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:38.105175 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:38.105959 (+ 784us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:45:38.105959 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:38.106008 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:45:38.106046 (+ 38us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:45:38.106262 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:45:38.106270 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:45:38.106274 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:45:38.106368 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 22:45:38.106531 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":117,"threads_started":1}
I20260806 22:45:38.107613 32466 heartbeater.cc:347] Connected to a master server at 127.18.45.254:41475
I20260806 22:45:38.107748 32466 heartbeater.cc:464] Registering TS with master...
I20260806 22:45:38.108110 32466 heartbeater.cc:511] Master 127.18.45.254:41475 requested a full tablet report, sending...
I20260806 22:45:38.108673 31978 ts_manager.cc:194] Registered new tserver with Master: 4c11993958054533b5ad8895935b3947 (127.18.45.194:34541)
I20260806 22:45:38.109218 31978 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.45.194:35033
W20260806 22:45:38.203740 32475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:45:38.204003 32475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:45:38.204025 32475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:45:38.205179 32475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:45:38.205214 32475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:45:38.205243 32475 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.195
I20260806 22:45:38.206555 32475 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:38745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.45.195
--webserver_port=0
--tserver_master_addrs=127.18.45.254:41475
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32475
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.195
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:45:38.206792 32475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:45:38.207194 32475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 22:45:38.213188 32475 server_base.cc:1034] running on GCE node
W20260806 22:45:38.213138 32482 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:38.213138 32481 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:38.213240 32485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:38.213675 32475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:45:38.214170 32475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:45:38.215397 32475 hybrid_clock.cc:648] HybridClock initialized: now 1786056338215373 us; error 39 us; skew 500 ppm
I20260806 22:45:38.217084 32475 webserver.cc:460] Webserver started at http://127.18.45.195:44365/ using document root <none> and password file <none>
I20260806 22:45:38.217497 32475 fs_manager.cc:359] Metadata directory not provided
I20260806 22:45:38.217581 32475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:45:38.217823 32475 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:45:38.219376 32475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-2/data/instance:
uuid: "9474c1d4ec714216bdeaed47095eb940"
format_stamp: "Formatted at 2026-08-06 22:45:38 on dist-test-slave-h3ft"
server_key: "370127dda4159679995cf5c22be8c1e0"
server_key_iv: "58712930b938133a012e6bc9e61de5a3"
server_key_version: "encryptionkey@0"
I20260806 22:45:38.219815 32475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-2/wal/instance:
uuid: "9474c1d4ec714216bdeaed47095eb940"
format_stamp: "Formatted at 2026-08-06 22:45:38 on dist-test-slave-h3ft"
server_key: "370127dda4159679995cf5c22be8c1e0"
server_key_iv: "58712930b938133a012e6bc9e61de5a3"
server_key_version: "encryptionkey@0"
I20260806 22:45:38.221423 32475 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:45:38.222357 32494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:38.222550 32475 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:45:38.222675 32475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-2/wal
uuid: "9474c1d4ec714216bdeaed47095eb940"
format_stamp: "Formatted at 2026-08-06 22:45:38 on dist-test-slave-h3ft"
server_key: "370127dda4159679995cf5c22be8c1e0"
server_key_iv: "58712930b938133a012e6bc9e61de5a3"
server_key_version: "encryptionkey@0"
I20260806 22:45:38.222765 32475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:45:38.246030 32475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:45:38.246500 32475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:45:38.246702 32475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:45:38.247098 32475 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:45:38.247541 32475 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:45:38.247601 32475 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:38.247663 32475 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:45:38.247694 32475 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:38.254913 32475 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.195:43591
I20260806 22:45:38.255013 32667 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.195:43591 every 8 connection(s)
I20260806 22:45:38.255440 32475 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
I20260806 22:45:38.261878 32037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:38.256350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.195:53051 (local address 127.18.45.254:41475)
0806 22:45:38.256422 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:38.256424 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:38.256733 (+ 309us) server_negotiation.cc:407] Connection header received
0806 22:45:38.257436 (+ 703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:38.257437 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:38.257461 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:38.257533 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:38.258336 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:38.258912 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:38.260038 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:38.260249 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:38.260338 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:38.261447 (+ 1109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:38.261449 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:38.261450 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:45:38.261456 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:38.261483 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:45:38.261487 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:38.261554 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:38.261748 (+ 194us) server_negotiation.cc:299] Negotiation successful
0806 22:45:38.261798 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 22:45:38.262130 32670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:38.256341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:41475 (local address 127.18.45.195:53051)
0806 22:45:38.256590 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:38.256632 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:38.257308 (+ 676us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:38.257621 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:38.257625 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:38.257634 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:38.258233 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:38.258237 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:38.259047 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:38.259048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:38.259914 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:38.259916 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:38.260346 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:38.260348 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:38.260510 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:38.261261 (+ 751us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:45:38.261262 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:38.261297 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:45:38.261324 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:45:38.261556 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:45:38.261564 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:45:38.261568 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:45:38.261720 (+ 152us) client_negotiation.cc:240] Negotiation successful
0806 22:45:38.261878 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":109,"threads_started":1}
I20260806 22:45:38.262962 32668 heartbeater.cc:347] Connected to a master server at 127.18.45.254:41475
I20260806 22:45:38.263136 32668 heartbeater.cc:464] Registering TS with master...
I20260806 22:45:38.263374 32668 heartbeater.cc:511] Master 127.18.45.254:41475 requested a full tablet report, sending...
I20260806 22:45:38.263885 31978 ts_manager.cc:194] Registered new tserver with Master: 9474c1d4ec714216bdeaed47095eb940 (127.18.45.195:43591)
I20260806 22:45:38.264070 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 32475
I20260806 22:45:38.264181 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-2/wal/instance
I20260806 22:45:38.264401 18615 external_mini_cluster.cc:1428] Setting key 1d2b0df78e3fbc53b376dfe801c2ebca
I20260806 22:45:38.264420 31978 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.45.195:53051
I20260806 22:45:38.265453 18615 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 22:45:38.267670 32037 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:45:38.267098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47658 (local address 127.18.45.254:41475)
0806 22:45:38.267178 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:38.267181 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:38.267256 (+ 75us) server_negotiation.cc:407] Connection header received
0806 22:45:38.267311 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:38.267311 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:38.267325 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:38.267396 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:38.267563 (+ 167us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47658: BlockingRecv error: recv got EOF from 127.0.0.1:47658 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 22:45:38.267690 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 32059
I20260806 22:45:38.282051 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 32264
I20260806 22:45:38.294111 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 32475
I20260806 22:45:38.307502 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 31935
2026-08-06T22:45:38Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (782 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-06T22:45:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:45:38Z Disabled control of system clock
I20260806 22:45:38.344326 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056275803658-18615-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056275803658-18615-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056275803658-18615-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:36731
--webserver_interface=127.18.45.254
--webserver_port=0
--builtin_ntp_servers=127.18.45.212:42675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.45.254:36731
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 22:45:38.442975 32681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:45:38.443264 32681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:45:38.443288 32681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:45:38.444465 32681 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:45:38.444501 32681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:45:38.444514 32681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:45:38.444527 32681 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:45:38.444540 32681 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:45:38.445884 32681 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:42675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056275803658-18615-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056275803658-18615-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.45.254:36731
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:36731
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.45.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32681
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056275803658-18615-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:45:38.446130 32681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:45:38.446429 32681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:45:38.451968 32688 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:38.451983 32692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:38.452039 32686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:38.452625 32681 server_base.cc:1034] running on GCE node
I20260806 22:45:38.452911 32681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:45:38.453236 32681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:45:38.454411 32681 hybrid_clock.cc:648] HybridClock initialized: now 1786056338454389 us; error 41 us; skew 500 ppm
I20260806 22:45:38.455968 32681 webserver.cc:460] Webserver started at http://127.18.45.254:45863/ using document root <none> and password file <none>
I20260806 22:45:38.456288 32681 fs_manager.cc:359] Metadata directory not provided
I20260806 22:45:38.456365 32681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:45:38.456526 32681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:45:38.457825 32681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056275803658-18615-0/minicluster-data/master-0/data/instance:
uuid: "f309b2a661c948fdb20a35cf57b5c129"
format_stamp: "Formatted at 2026-08-06 22:45:38 on dist-test-slave-h3ft"
server_key: "381d1578cc887295a14a21da67ce6204"
server_key_iv: "b556e88969a9df2ae4174a321027cbbf"
server_key_version: "encryptionkey@0"
I20260806 22:45:38.458292 32681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056275803658-18615-0/minicluster-data/master-0/wal/instance:
uuid: "f309b2a661c948fdb20a35cf57b5c129"
format_stamp: "Formatted at 2026-08-06 22:45:38 on dist-test-slave-h3ft"
server_key: "381d1578cc887295a14a21da67ce6204"
server_key_iv: "b556e88969a9df2ae4174a321027cbbf"
server_key_version: "encryptionkey@0"
I20260806 22:45:38.460069 32681 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260806 22:45:38.460877 32703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:38.461079 32681 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:45:38.461180 32681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056275803658-18615-0/minicluster-data/master-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056275803658-18615-0/minicluster-data/master-0/wal
uuid: "f309b2a661c948fdb20a35cf57b5c129"
format_stamp: "Formatted at 2026-08-06 22:45:38 on dist-test-slave-h3ft"
server_key: "381d1578cc887295a14a21da67ce6204"
server_key_iv: "b556e88969a9df2ae4174a321027cbbf"
server_key_version: "encryptionkey@0"
I20260806 22:45:38.461265 32681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056275803658-18615-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056275803658-18615-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056275803658-18615-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:45:38.465096 32681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:45:38.465402 32681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:45:38.465559 32681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:45:38.470396 32681 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.254:36731
I20260806 22:45:38.470433 319 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.254:36731 every 8 connection(s)
I20260806 22:45:38.470822 32681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
I20260806 22:45:38.471692 321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:45:38.474996 321 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f309b2a661c948fdb20a35cf57b5c129: Bootstrap starting.
I20260806 22:45:38.475860 321 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f309b2a661c948fdb20a35cf57b5c129: Neither blocks nor log segments found. Creating new log.
I20260806 22:45:38.476222 321 log.cc:824] T 00000000000000000000000000000000 P f309b2a661c948fdb20a35cf57b5c129: Log is configured to *not* fsync() on all Append() calls
I20260806 22:45:38.477022 321 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f309b2a661c948fdb20a35cf57b5c129: No bootstrap required, opened a new log
I20260806 22:45:38.478849 321 raft_consensus.cc:348] T 00000000000000000000000000000000 P f309b2a661c948fdb20a35cf57b5c129 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f309b2a661c948fdb20a35cf57b5c129" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 36731 } }
I20260806 22:45:38.479067 321 raft_consensus.cc:374] T 00000000000000000000000000000000 P f309b2a661c948fdb20a35cf57b5c129 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:45:38.479147 321 raft_consensus.cc:729] T 00000000000000000000000000000000 P f309b2a661c948fdb20a35cf57b5c129 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f309b2a661c948fdb20a35cf57b5c129, State: Initialized, Role: FOLLOWER
I20260806 22:45:38.479324 321 consensus_queue.cc:260] T 00000000000000000000000000000000 P f309b2a661c948fdb20a35cf57b5c129 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f309b2a661c948fdb20a35cf57b5c129" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 36731 } }
I20260806 22:45:38.479399 321 raft_consensus.cc:388] T 00000000000000000000000000000000 P f309b2a661c948fdb20a35cf57b5c129 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:45:38.479444 321 raft_consensus.cc:482] T 00000000000000000000000000000000 P f309b2a661c948fdb20a35cf57b5c129 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:45:38.479506 321 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f309b2a661c948fdb20a35cf57b5c129 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:45:38.479851 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 32681
I20260806 22:45:38.479976 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056275803658-18615-0/minicluster-data/master-0/wal/instance
I20260806 22:45:38.480213 18615 external_mini_cluster.cc:1428] Setting key 12373f52e6a258bf8b600bf04de4482e
I20260806 22:45:38.480193 321 raft_consensus.cc:504] T 00000000000000000000000000000000 P f309b2a661c948fdb20a35cf57b5c129 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f309b2a661c948fdb20a35cf57b5c129" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 36731 } }
I20260806 22:45:38.480326 321 leader_election.cc:302] T 00000000000000000000000000000000 P f309b2a661c948fdb20a35cf57b5c129 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f309b2a661c948fdb20a35cf57b5c129; no voters:
I20260806 22:45:38.480561 321 leader_election.cc:288] T 00000000000000000000000000000000 P f309b2a661c948fdb20a35cf57b5c129 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:45:38.480597 324 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f309b2a661c948fdb20a35cf57b5c129 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:45:38.480733 324 raft_consensus.cc:686] T 00000000000000000000000000000000 P f309b2a661c948fdb20a35cf57b5c129 [term 1 LEADER]: Becoming Leader. State: Replica: f309b2a661c948fdb20a35cf57b5c129, State: Running, Role: LEADER
I20260806 22:45:38.480887 321 sys_catalog.cc:564] T 00000000000000000000000000000000 P f309b2a661c948fdb20a35cf57b5c129 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:45:38.480892 324 consensus_queue.cc:237] T 00000000000000000000000000000000 P f309b2a661c948fdb20a35cf57b5c129 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f309b2a661c948fdb20a35cf57b5c129" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 36731 } }
I20260806 22:45:38.481348 324 sys_catalog.cc:455] T 00000000000000000000000000000000 P f309b2a661c948fdb20a35cf57b5c129 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f309b2a661c948fdb20a35cf57b5c129. Latest consensus state: current_term: 1 leader_uuid: "f309b2a661c948fdb20a35cf57b5c129" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f309b2a661c948fdb20a35cf57b5c129" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 36731 } } }
I20260806 22:45:38.481513 324 sys_catalog.cc:458] T 00000000000000000000000000000000 P f309b2a661c948fdb20a35cf57b5c129 [sys.catalog]: This master's current role is: LEADER
I20260806 22:45:38.481738 325 sys_catalog.cc:455] T 00000000000000000000000000000000 P f309b2a661c948fdb20a35cf57b5c129 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f309b2a661c948fdb20a35cf57b5c129" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f309b2a661c948fdb20a35cf57b5c129" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 36731 } } }
I20260806 22:45:38.481824 325 sys_catalog.cc:458] T 00000000000000000000000000000000 P f309b2a661c948fdb20a35cf57b5c129 [sys.catalog]: This master's current role is: LEADER
I20260806 22:45:38.481830 336 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:45:38.482846 336 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:45:38.483758 328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:38.481036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37086 (local address 127.18.45.254:36731)
0806 22:45:38.481307 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:38.481317 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:38.481353 (+ 36us) server_negotiation.cc:407] Connection header received
0806 22:45:38.481485 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:38.481493 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:38.481549 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:38.481708 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:38.483189 (+ 1481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:38.483194 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:38.483194 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:45:38.483199 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:38.483248 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:45:38.483264 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:38.483332 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:38.483490 (+ 158us) server_negotiation.cc:299] Negotiation successful
0806 22:45:38.483562 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":110,"threads_started":1}
I20260806 22:45:38.484887 336 catalog_manager.cc:1347] Generated new cluster ID: 5ec67659ab3d4d94972daf3c7f65b9a6
I20260806 22:45:38.484943 336 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:45:38.499267 336 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:45:38.499507 336 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:45:38.501981 336 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f309b2a661c948fdb20a35cf57b5c129: Generated new TSK 0
I20260806 22:45:38.502223 336 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:45:38.504688 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.193:0
--local_ip_for_outbound_sockets=127.18.45.193
--webserver_interface=127.18.45.193
--webserver_port=0
--tserver_master_addrs=127.18.45.254:36731
--builtin_ntp_servers=127.18.45.212:42675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 22:45:38.605299 351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:45:38.605613 351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:45:38.605659 351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:45:38.607121 351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:45:38.607189 351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:45:38.607235 351 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.193
I20260806 22:45:38.608795 351 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:42675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.45.193
--webserver_port=0
--tserver_master_addrs=127.18.45.254:36731
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.351
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.193
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:45:38.609122 351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:45:38.609498 351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:45:38.616626 360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:38.616581 361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:38.616572 364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:38.617132 351 server_base.cc:1034] running on GCE node
I20260806 22:45:38.617440 351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:45:38.617851 351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:45:38.619077 351 hybrid_clock.cc:648] HybridClock initialized: now 1786056338619054 us; error 49 us; skew 500 ppm
I20260806 22:45:38.620915 351 webserver.cc:460] Webserver started at http://127.18.45.193:35789/ using document root <none> and password file <none>
I20260806 22:45:38.621234 351 fs_manager.cc:359] Metadata directory not provided
I20260806 22:45:38.621301 351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:45:38.621469 351 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:45:38.623358 351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-0/data/instance:
uuid: "51453a70d5b8423d892366eba725cf38"
format_stamp: "Formatted at 2026-08-06 22:45:38 on dist-test-slave-h3ft"
server_key: "4217ee7ade06851a192a041391eb852c"
server_key_iv: "cfb180f049d3eb35d3c8aa6ec6e581a1"
server_key_version: "encryptionkey@0"
I20260806 22:45:38.623939 351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-0/wal/instance:
uuid: "51453a70d5b8423d892366eba725cf38"
format_stamp: "Formatted at 2026-08-06 22:45:38 on dist-test-slave-h3ft"
server_key: "4217ee7ade06851a192a041391eb852c"
server_key_iv: "cfb180f049d3eb35d3c8aa6ec6e581a1"
server_key_version: "encryptionkey@0"
I20260806 22:45:38.625983 351 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 22:45:38.627075 375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:38.627295 351 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:45:38.627453 351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-0/wal
uuid: "51453a70d5b8423d892366eba725cf38"
format_stamp: "Formatted at 2026-08-06 22:45:38 on dist-test-slave-h3ft"
server_key: "4217ee7ade06851a192a041391eb852c"
server_key_iv: "cfb180f049d3eb35d3c8aa6ec6e581a1"
server_key_version: "encryptionkey@0"
I20260806 22:45:38.627565 351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:45:38.633111 351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:45:38.633498 351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:45:38.633699 351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:45:38.633955 351 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:45:38.634298 351 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:45:38.634334 351 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:38.634369 351 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:45:38.634383 351 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:38.642261 351 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.193:40581
I20260806 22:45:38.642300 532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.193:40581 every 8 connection(s)
I20260806 22:45:38.642778 351 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
I20260806 22:45:38.645262 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 351
I20260806 22:45:38.645412 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-0/wal/instance
I20260806 22:45:38.645632 18615 external_mini_cluster.cc:1428] Setting key 683dc450f42caf3033002e39bbc1af06
I20260806 22:45:38.646901 328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:38.643777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:48767 (local address 127.18.45.254:36731)
0806 22:45:38.643849 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:38.643852 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:38.644152 (+ 300us) server_negotiation.cc:407] Connection header received
0806 22:45:38.644783 (+ 631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:38.644784 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:38.644796 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:38.644865 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:38.646526 (+ 1661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:38.646527 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:38.646528 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:45:38.646532 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:38.646551 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:45:38.646556 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:38.646651 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:38.646754 (+ 103us) server_negotiation.cc:299] Negotiation successful
0806 22:45:38.646775 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 22:45:38.647002 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.194:0
--local_ip_for_outbound_sockets=127.18.45.194
--webserver_interface=127.18.45.194
--webserver_port=0
--tserver_master_addrs=127.18.45.254:36731
--builtin_ntp_servers=127.18.45.212:42675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260806 22:45:38.647058 538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:38.643757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:36731 (local address 127.18.45.193:48767)
0806 22:45:38.644001 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:38.644047 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:38.644663 (+ 616us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:38.644957 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:38.644978 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:38.644987 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:38.646317 (+ 1330us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:45:38.646318 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:38.646362 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:45:38.646405 (+ 43us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:45:38.646664 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:45:38.646669 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:45:38.646672 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:45:38.646742 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 22:45:38.646804 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":117,"threads_started":1}
I20260806 22:45:38.647581 533 heartbeater.cc:347] Connected to a master server at 127.18.45.254:36731
I20260806 22:45:38.647702 533 heartbeater.cc:464] Registering TS with master...
I20260806 22:45:38.647876 533 heartbeater.cc:511] Master 127.18.45.254:36731 requested a full tablet report, sending...
I20260806 22:45:38.648496 32730 ts_manager.cc:194] Registered new tserver with Master: 51453a70d5b8423d892366eba725cf38 (127.18.45.193:40581)
W20260806 22:45:38.744776 540 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:45:38.745035 540 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:45:38.745061 540 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:45:38.746388 540 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:45:38.746438 540 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:45:38.746467 540 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.194
I20260806 22:45:38.747991 540 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:42675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.45.194
--webserver_port=0
--tserver_master_addrs=127.18.45.254:36731
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.540
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.194
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:45:38.748301 540 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:45:38.748687 540 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:45:38.754748 547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:38.754748 551 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:38.754861 549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:38.755321 540 server_base.cc:1034] running on GCE node
I20260806 22:45:38.755582 540 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:45:38.755837 540 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:45:38.756999 540 hybrid_clock.cc:648] HybridClock initialized: now 1786056338756970 us; error 40 us; skew 500 ppm
I20260806 22:45:38.758585 540 webserver.cc:460] Webserver started at http://127.18.45.194:46445/ using document root <none> and password file <none>
I20260806 22:45:38.758864 540 fs_manager.cc:359] Metadata directory not provided
I20260806 22:45:38.758916 540 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:45:38.759143 540 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:45:38.760468 540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-1/data/instance:
uuid: "46dec81cbd814ac689ee325573edbe68"
format_stamp: "Formatted at 2026-08-06 22:45:38 on dist-test-slave-h3ft"
server_key: "5ac8dc648dc8421803ae3313ae00fd9a"
server_key_iv: "f2f7ee39fc2c0309c44cd6105aa8f5b3"
server_key_version: "encryptionkey@0"
I20260806 22:45:38.760896 540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-1/wal/instance:
uuid: "46dec81cbd814ac689ee325573edbe68"
format_stamp: "Formatted at 2026-08-06 22:45:38 on dist-test-slave-h3ft"
server_key: "5ac8dc648dc8421803ae3313ae00fd9a"
server_key_iv: "f2f7ee39fc2c0309c44cd6105aa8f5b3"
server_key_version: "encryptionkey@0"
I20260806 22:45:38.762423 540 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:45:38.763540 558 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:38.763720 540 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:45:38.763842 540 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-1/wal
uuid: "46dec81cbd814ac689ee325573edbe68"
format_stamp: "Formatted at 2026-08-06 22:45:38 on dist-test-slave-h3ft"
server_key: "5ac8dc648dc8421803ae3313ae00fd9a"
server_key_iv: "f2f7ee39fc2c0309c44cd6105aa8f5b3"
server_key_version: "encryptionkey@0"
I20260806 22:45:38.763928 540 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:45:38.767911 540 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:45:38.768219 540 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:45:38.768374 540 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:45:38.768612 540 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:45:38.768960 540 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:45:38.769021 540 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:38.769066 540 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:45:38.769091 540 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:38.776206 540 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.194:42067
I20260806 22:45:38.776262 731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.194:42067 every 8 connection(s)
I20260806 22:45:38.776684 540 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
I20260806 22:45:38.780526 328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:38.777657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:42109 (local address 127.18.45.254:36731)
0806 22:45:38.777759 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:38.777762 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:38.778011 (+ 249us) server_negotiation.cc:407] Connection header received
0806 22:45:38.778675 (+ 664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:38.778676 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:38.778688 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:38.778744 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:38.780177 (+ 1433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:38.780179 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:38.780179 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:45:38.780184 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:38.780207 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:45:38.780215 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:38.780297 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:38.780414 (+ 117us) server_negotiation.cc:299] Negotiation successful
0806 22:45:38.780435 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 22:45:38.780783 734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:38.777628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:36731 (local address 127.18.45.194:42109)
0806 22:45:38.777864 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:38.777905 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:38.778541 (+ 636us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:38.778837 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:38.778841 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:38.778851 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:38.779973 (+ 1122us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:45:38.779974 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:38.780007 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:45:38.780042 (+ 35us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:45:38.780314 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:45:38.780323 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:45:38.780331 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 22:45:38.780427 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 22:45:38.780514 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":92,"threads_started":1}
I20260806 22:45:38.781507 732 heartbeater.cc:347] Connected to a master server at 127.18.45.254:36731
I20260806 22:45:38.781625 732 heartbeater.cc:464] Registering TS with master...
I20260806 22:45:38.781816 732 heartbeater.cc:511] Master 127.18.45.254:36731 requested a full tablet report, sending...
I20260806 22:45:38.782331 32730 ts_manager.cc:194] Registered new tserver with Master: 46dec81cbd814ac689ee325573edbe68 (127.18.45.194:42067)
I20260806 22:45:38.784854 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 540
I20260806 22:45:38.784998 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-1/wal/instance
I20260806 22:45:38.785192 18615 external_mini_cluster.cc:1428] Setting key 70e2f64ea7e2683229841939842ad7b0
I20260806 22:45:38.786329 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.195:0
--local_ip_for_outbound_sockets=127.18.45.195
--webserver_interface=127.18.45.195
--webserver_port=0
--tserver_master_addrs=127.18.45.254:36731
--builtin_ntp_servers=127.18.45.212:42675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 22:45:38.890396 735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:45:38.890656 735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:45:38.890683 735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:45:38.892045 735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:45:38.892094 735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:45:38.892122 735 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.195
I20260806 22:45:38.893620 735 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:42675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.45.195
--webserver_port=0
--tserver_master_addrs=127.18.45.254:36731
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.735
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.195
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:45:38.893940 735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:45:38.894325 735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:45:38.900271 744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:38.900271 749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:38.900327 735 server_base.cc:1034] running on GCE node
W20260806 22:45:38.900285 745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:38.900823 735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:45:38.901113 735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:45:38.902264 735 hybrid_clock.cc:648] HybridClock initialized: now 1786056338902245 us; error 50 us; skew 500 ppm
I20260806 22:45:38.903868 735 webserver.cc:460] Webserver started at http://127.18.45.195:42251/ using document root <none> and password file <none>
I20260806 22:45:38.904206 735 fs_manager.cc:359] Metadata directory not provided
I20260806 22:45:38.904260 735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:45:38.904462 735 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:45:38.905767 735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-2/data/instance:
uuid: "c257b22ae1194c5eae9a29d89939a64c"
format_stamp: "Formatted at 2026-08-06 22:45:38 on dist-test-slave-h3ft"
server_key: "614f9124686b22239949cd8798b11b02"
server_key_iv: "a20c0c3d137f83f42652e1ff07293151"
server_key_version: "encryptionkey@0"
I20260806 22:45:38.906188 735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-2/wal/instance:
uuid: "c257b22ae1194c5eae9a29d89939a64c"
format_stamp: "Formatted at 2026-08-06 22:45:38 on dist-test-slave-h3ft"
server_key: "614f9124686b22239949cd8798b11b02"
server_key_iv: "a20c0c3d137f83f42652e1ff07293151"
server_key_version: "encryptionkey@0"
I20260806 22:45:38.907816 735 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260806 22:45:38.908739 761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:38.908949 735 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:45:38.909070 735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-2/wal
uuid: "c257b22ae1194c5eae9a29d89939a64c"
format_stamp: "Formatted at 2026-08-06 22:45:38 on dist-test-slave-h3ft"
server_key: "614f9124686b22239949cd8798b11b02"
server_key_iv: "a20c0c3d137f83f42652e1ff07293151"
server_key_version: "encryptionkey@0"
I20260806 22:45:38.909157 735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:45:38.913241 735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:45:38.913597 735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:45:38.913765 735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:45:38.914006 735 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:45:38.914398 735 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:45:38.914465 735 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:38.914516 735 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:45:38.914549 735 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:38.921891 735 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.195:45677
I20260806 22:45:38.921947 929 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.195:45677 every 8 connection(s)
I20260806 22:45:38.922369 735 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
I20260806 22:45:38.926011 328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:38.923353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.195:36197 (local address 127.18.45.254:36731)
0806 22:45:38.923458 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:38.923461 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:38.923732 (+ 271us) server_negotiation.cc:407] Connection header received
0806 22:45:38.924313 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:38.924314 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:38.924325 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:38.924404 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:38.925737 (+ 1333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:38.925739 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:38.925739 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:45:38.925744 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:38.925763 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:45:38.925769 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:38.925861 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:38.925923 (+ 62us) server_negotiation.cc:299] Negotiation successful
0806 22:45:38.925939 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 22:45:38.926307 932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:38.923301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:36731 (local address 127.18.45.195:36197)
0806 22:45:38.923541 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:38.923591 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:38.924186 (+ 595us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:38.924468 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:38.924472 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:38.924482 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:38.925551 (+ 1069us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:45:38.925551 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:38.925586 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:45:38.925623 (+ 37us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:45:38.925841 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:45:38.925847 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:45:38.925850 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:45:38.925916 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 22:45:38.925977 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":120,"threads_started":1}
I20260806 22:45:38.926285 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 735
I20260806 22:45:38.926405 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786056275803658-18615-0/minicluster-data/ts-2/wal/instance
I20260806 22:45:38.926667 18615 external_mini_cluster.cc:1428] Setting key 4b65bb0e42410809b363e7adb29b3128
I20260806 22:45:38.926985 930 heartbeater.cc:347] Connected to a master server at 127.18.45.254:36731
I20260806 22:45:38.927150 930 heartbeater.cc:464] Registering TS with master...
I20260806 22:45:38.927330 930 heartbeater.cc:511] Master 127.18.45.254:36731 requested a full tablet report, sending...
I20260806 22:45:38.927829 32730 ts_manager.cc:194] Registered new tserver with Master: c257b22ae1194c5eae9a29d89939a64c (127.18.45.195:45677)
I20260806 22:45:38.939373 18615 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 22:45:38.941817 328 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:45:38.941268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40650 (local address 127.18.45.254:36731)
0806 22:45:38.941342 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:38.941344 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:38.941406 (+ 62us) server_negotiation.cc:407] Connection header received
0806 22:45:38.941476 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:38.941477 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:38.941483 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:38.941545 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:38.941714 (+ 169us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40650: BlockingRecv error: recv got EOF from 127.0.0.1:40650 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 22:45:38.941887 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 351
I20260806 22:45:38.957191 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 540
I20260806 22:45:38.974295 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 735
I20260806 22:45:38.988266 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 32681
2026-08-06T22:45:38Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (680 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:45:39 dist-test-slave-h3ft krb5kdc[945](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:45:39 dist-test-slave-h3ft krb5kdc[945](info): set up 2 sockets
Aug 06 22:45:39 dist-test-slave-h3ft krb5kdc[945](info): commencing operation
krb5kdc: starting...
W20260806 22:45:40.322504 18615 mini_kdc.cc:121] Time spent starting KDC: real 1.307s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Aug 06 22:45:40 dist-test-slave-h3ft krb5kdc[945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056340, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:45:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:45:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.45.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab.
Entry for principal kudu/127.18.45.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab.
I20260806 22:45:40.462047 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:41319
--webserver_interface=127.18.45.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:43239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.45.254:41319
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:45:40.556257 964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:45:40.556526 964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:45:40.556552 964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:45:40.557821 964 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:45:40.557861 964 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:45:40.557876 964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:45:40.557889 964 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:45:40.557902 964 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:45:40.559458 964 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:43239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.45.254:41319
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:41319
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.45.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.964
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:45:40.559794 964 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:45:40.560101 964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:45:40.565651 974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:40.565662 978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:40.565709 971 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:40.565963 964 server_base.cc:1034] running on GCE node
I20260806 22:45:40.566370 964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:45:40.566720 964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:45:40.567894 964 hybrid_clock.cc:648] HybridClock initialized: now 1786056340567864 us; error 77 us; skew 500 ppm
Aug 06 22:45:40 dist-test-slave-h3ft krb5kdc[945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056340, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:45:40.576422 964 init.cc:377] Logged in from keytab as kudu/127.18.45.254@KRBTEST.COM (short username kudu)
I20260806 22:45:40.576970 964 webserver.cc:460] Webserver started at http://127.18.45.254:45217/ using document root <none> and password file <none>
I20260806 22:45:40.577245 964 fs_manager.cc:359] Metadata directory not provided
I20260806 22:45:40.577322 964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:45:40.577492 964 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:45:40.578737 964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/minicluster-data/master-0/data/instance:
uuid: "ee605b5e78f54a7584404506992c376e"
format_stamp: "Formatted at 2026-08-06 22:45:40 on dist-test-slave-h3ft"
server_key: "88301d8d5951a4ed396693068773bea0"
server_key_iv: "42429974778d34bf90818deea789d806"
server_key_version: "encryptionkey@0"
I20260806 22:45:40.579232 964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/minicluster-data/master-0/wal/instance:
uuid: "ee605b5e78f54a7584404506992c376e"
format_stamp: "Formatted at 2026-08-06 22:45:40 on dist-test-slave-h3ft"
server_key: "88301d8d5951a4ed396693068773bea0"
server_key_iv: "42429974778d34bf90818deea789d806"
server_key_version: "encryptionkey@0"
I20260806 22:45:40.580773 964 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:45:40.581689 988 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:40.581933 964 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:45:40.582060 964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/minicluster-data/master-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/minicluster-data/master-0/wal
uuid: "ee605b5e78f54a7584404506992c376e"
format_stamp: "Formatted at 2026-08-06 22:45:40 on dist-test-slave-h3ft"
server_key: "88301d8d5951a4ed396693068773bea0"
server_key_iv: "42429974778d34bf90818deea789d806"
server_key_version: "encryptionkey@0"
I20260806 22:45:40.582146 964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:45:40.594548 964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:45:40.596849 964 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:45:40.597047 964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:45:40.602005 964 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.254:41319
I20260806 22:45:40.602041 1064 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.254:41319 every 8 connection(s)
I20260806 22:45:40.602487 964 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
I20260806 22:45:40.603777 1067 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:45:40.606554 1067 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee605b5e78f54a7584404506992c376e: Bootstrap starting.
I20260806 22:45:40.607331 1067 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ee605b5e78f54a7584404506992c376e: Neither blocks nor log segments found. Creating new log.
I20260806 22:45:40.607686 1067 log.cc:824] T 00000000000000000000000000000000 P ee605b5e78f54a7584404506992c376e: Log is configured to *not* fsync() on all Append() calls
I20260806 22:45:40.608543 1067 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee605b5e78f54a7584404506992c376e: No bootstrap required, opened a new log
I20260806 22:45:40.608880 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 964
I20260806 22:45:40.609032 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/minicluster-data/master-0/wal/instance
I20260806 22:45:40.609257 18615 external_mini_cluster.cc:1428] Setting key a21a37a7737b8ec7134cb92cad59948a
I20260806 22:45:40.611294 1067 raft_consensus.cc:348] T 00000000000000000000000000000000 P ee605b5e78f54a7584404506992c376e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee605b5e78f54a7584404506992c376e" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 41319 } }
I20260806 22:45:40.611482 1067 raft_consensus.cc:374] T 00000000000000000000000000000000 P ee605b5e78f54a7584404506992c376e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:45:40.611531 1067 raft_consensus.cc:729] T 00000000000000000000000000000000 P ee605b5e78f54a7584404506992c376e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee605b5e78f54a7584404506992c376e, State: Initialized, Role: FOLLOWER
I20260806 22:45:40.611680 1067 consensus_queue.cc:260] T 00000000000000000000000000000000 P ee605b5e78f54a7584404506992c376e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee605b5e78f54a7584404506992c376e" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 41319 } }
I20260806 22:45:40.611776 1067 raft_consensus.cc:388] T 00000000000000000000000000000000 P ee605b5e78f54a7584404506992c376e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:45:40.611824 1067 raft_consensus.cc:482] T 00000000000000000000000000000000 P ee605b5e78f54a7584404506992c376e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:45:40.611883 1067 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ee605b5e78f54a7584404506992c376e [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:45:40.612505 1067 raft_consensus.cc:504] T 00000000000000000000000000000000 P ee605b5e78f54a7584404506992c376e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee605b5e78f54a7584404506992c376e" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 41319 } }
I20260806 22:45:40.612653 1067 leader_election.cc:302] T 00000000000000000000000000000000 P ee605b5e78f54a7584404506992c376e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ee605b5e78f54a7584404506992c376e; no voters:
I20260806 22:45:40.612872 1067 leader_election.cc:288] T 00000000000000000000000000000000 P ee605b5e78f54a7584404506992c376e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:45:40.612913 1074 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ee605b5e78f54a7584404506992c376e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:45:40.613057 1074 raft_consensus.cc:686] T 00000000000000000000000000000000 P ee605b5e78f54a7584404506992c376e [term 1 LEADER]: Becoming Leader. State: Replica: ee605b5e78f54a7584404506992c376e, State: Running, Role: LEADER
I20260806 22:45:40.613152 1074 consensus_queue.cc:237] T 00000000000000000000000000000000 P ee605b5e78f54a7584404506992c376e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee605b5e78f54a7584404506992c376e" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 41319 } }
I20260806 22:45:40.613198 1067 sys_catalog.cc:564] T 00000000000000000000000000000000 P ee605b5e78f54a7584404506992c376e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:45:40.613555 1075 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee605b5e78f54a7584404506992c376e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ee605b5e78f54a7584404506992c376e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee605b5e78f54a7584404506992c376e" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 41319 } } }
I20260806 22:45:40.613647 1075 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee605b5e78f54a7584404506992c376e [sys.catalog]: This master's current role is: LEADER
I20260806 22:45:40.613914 1076 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee605b5e78f54a7584404506992c376e [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee605b5e78f54a7584404506992c376e. Latest consensus state: current_term: 1 leader_uuid: "ee605b5e78f54a7584404506992c376e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee605b5e78f54a7584404506992c376e" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 41319 } } }
I20260806 22:45:40.613986 1076 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee605b5e78f54a7584404506992c376e [sys.catalog]: This master's current role is: LEADER
I20260806 22:45:40.614269 1080 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:45:40.614965 1080 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:45:40.617547 1080 catalog_manager.cc:1347] Generated new cluster ID: 2c529ea38a6b4dd6a9cb1cd884ac156b
I20260806 22:45:40.617607 1080 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:45:40 dist-test-slave-h3ft krb5kdc[945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056340, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:45:40.622718 1073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:40.609944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41060 (local address 127.18.45.254:41319)
0806 22:45:40.610202 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:40.610212 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:40.610241 (+ 29us) server_negotiation.cc:407] Connection header received
0806 22:45:40.611225 (+ 984us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:40.611234 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:40.611292 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:40.611460 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:40.612306 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:40.613605 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:40.614991 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:40.615706 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:40.615823 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:40.619121 (+ 3298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:40.619129 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:40.619129 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:40.619145 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:40.621041 (+ 1896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:40.621457 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:40.621461 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:40.621461 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:40.621561 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:40.621784 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:40.621786 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:40.621786 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:40.622057 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:40.622262 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:40.622335 (+ 73us) server_negotiation.cc:299] Negotiation successful
0806 22:45:40.622490 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":128,"threads_started":1}
I20260806 22:45:40.641157 1080 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:45:40.641805 1080 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:45:40.647691 1080 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ee605b5e78f54a7584404506992c376e: Generated new TSK 0
I20260806 22:45:40.647907 1080 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.45.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab.
Entry for principal kudu/127.18.45.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab.
I20260806 22:45:40.682893 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.193:0
--local_ip_for_outbound_sockets=127.18.45.193
--webserver_interface=127.18.45.193
--webserver_port=0
--tserver_master_addrs=127.18.45.254:41319
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab
--principal=kudu/127.18.45.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:43239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:45:40.779219 1113 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:45:40.779510 1113 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:45:40.779551 1113 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:45:40.780843 1113 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:45:40.780910 1113 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:45:40.780957 1113 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.193
I20260806 22:45:40.782629 1113 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:43239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab
--principal=kudu/127.18.45.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.45.193
--webserver_port=0
--tserver_master_addrs=127.18.45.254:41319
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1113
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.193
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:45:40.782934 1113 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:45:40.783263 1113 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:45:40.789294 1121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:40.789304 1122 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:40.789316 1124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:40.789592 1113 server_base.cc:1034] running on GCE node
I20260806 22:45:40.789878 1113 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:45:40.790109 1113 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:45:40.791256 1113 hybrid_clock.cc:648] HybridClock initialized: now 1786056340791244 us; error 33 us; skew 500 ppm
Aug 06 22:45:40 dist-test-slave-h3ft krb5kdc[945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056340, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:45:40.794199 1113 init.cc:377] Logged in from keytab as kudu/127.18.45.193@KRBTEST.COM (short username kudu)
I20260806 22:45:40.794726 1113 webserver.cc:460] Webserver started at http://127.18.45.193:42849/ using document root <none> and password file <none>
I20260806 22:45:40.794984 1113 fs_manager.cc:359] Metadata directory not provided
I20260806 22:45:40.795091 1113 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:45:40.795276 1113 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:45:40.796598 1113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/minicluster-data/ts-0/data/instance:
uuid: "cef903805dbc4802bf13d0036778c46b"
format_stamp: "Formatted at 2026-08-06 22:45:40 on dist-test-slave-h3ft"
server_key: "166d15a852db70176976774f2c0f3ff7"
server_key_iv: "b1e70518c72d4e0931f4751b728f9182"
server_key_version: "encryptionkey@0"
I20260806 22:45:40.797048 1113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/minicluster-data/ts-0/wal/instance:
uuid: "cef903805dbc4802bf13d0036778c46b"
format_stamp: "Formatted at 2026-08-06 22:45:40 on dist-test-slave-h3ft"
server_key: "166d15a852db70176976774f2c0f3ff7"
server_key_iv: "b1e70518c72d4e0931f4751b728f9182"
server_key_version: "encryptionkey@0"
I20260806 22:45:40.798498 1113 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:45:40.799368 1134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:40.799556 1113 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:45:40.799674 1113 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/minicluster-data/ts-0/wal
uuid: "cef903805dbc4802bf13d0036778c46b"
format_stamp: "Formatted at 2026-08-06 22:45:40 on dist-test-slave-h3ft"
server_key: "166d15a852db70176976774f2c0f3ff7"
server_key_iv: "b1e70518c72d4e0931f4751b728f9182"
server_key_version: "encryptionkey@0"
I20260806 22:45:40.799754 1113 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:45:40.821942 1113 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:45:40.824540 1113 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:45:40.824725 1113 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:45:40.825050 1113 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:45:40.825454 1113 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:45:40.825516 1113 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:40.825568 1113 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:45:40.825599 1113 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:40.833003 1113 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.193:43103
I20260806 22:45:40.833105 1311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.193:43103 every 8 connection(s)
I20260806 22:45:40.833492 1113 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
I20260806 22:45:40.840795 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 1113
I20260806 22:45:40.840947 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/minicluster-data/ts-0/wal/instance
Aug 06 22:45:40 dist-test-slave-h3ft krb5kdc[945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056340, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.193@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:45:40.841212 18615 external_mini_cluster.cc:1428] Setting key 3c473f8278f15a3d435c5d65062515dd
I20260806 22:45:40.846589 1073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:40.834433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:57647 (local address 127.18.45.254:41319)
0806 22:45:40.834543 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:40.834546 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:40.834754 (+ 208us) server_negotiation.cc:407] Connection header received
0806 22:45:40.835539 (+ 785us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:40.835541 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:40.835556 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:40.835624 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:40.836885 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:40.837482 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:40.838560 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:40.838791 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:40.838857 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:40.841856 (+ 2999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:40.841858 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:40.841858 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:40.841866 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:40.844163 (+ 2297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:40.844877 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:40.844878 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:40.844879 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:40.844984 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:40.845429 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:40.845430 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:40.845431 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:40.845649 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:40.845733 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:40.846235 (+ 502us) server_negotiation.cc:299] Negotiation successful
0806 22:45:40.846425 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 22:45:40.846712 1317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:40.834375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:41319 (local address 127.18.45.193:57647)
0806 22:45:40.834609 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:40.834654 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:40.835352 (+ 698us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:40.835776 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:40.835781 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:40.836525 (+ 744us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:40.836772 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:40.836777 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:40.837572 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:40.837573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:40.838465 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:40.838467 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:40.838881 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:40.838882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:40.839088 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:40.839689 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:40.839700 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:40.841739 (+ 2039us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:45:40.844276 (+ 2537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:40.844288 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:40.844291 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:40.844775 (+ 484us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:40.845077 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:40.845078 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:40.845079 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:40.845277 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:40.845740 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:45:40.845748 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:45:40.845975 (+ 227us) client_negotiation.cc:769] Sending connection context
0806 22:45:40.846183 (+ 208us) client_negotiation.cc:240] Negotiation successful
0806 22:45:40.846418 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":119,"threads_started":1}
I20260806 22:45:40.848233 1314 heartbeater.cc:347] Connected to a master server at 127.18.45.254:41319
I20260806 22:45:40.848505 1314 heartbeater.cc:464] Registering TS with master...
I20260806 22:45:40.848850 1314 heartbeater.cc:511] Master 127.18.45.254:41319 requested a full tablet report, sending...
I20260806 22:45:40.849699 1012 ts_manager.cc:194] Registered new tserver with Master: cef903805dbc4802bf13d0036778c46b (127.18.45.193:43103)
I20260806 22:45:40.850795 1012 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.45.193@KRBTEST.COM'} at 127.18.45.193:57647
WARNING: no policy specified for kudu/127.18.45.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab.
Entry for principal kudu/127.18.45.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab.
I20260806 22:45:40.884130 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.194:0
--local_ip_for_outbound_sockets=127.18.45.194
--webserver_interface=127.18.45.194
--webserver_port=0
--tserver_master_addrs=127.18.45.254:41319
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab
--principal=kudu/127.18.45.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:43239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:45:40.984086 1322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:45:40.984587 1322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:45:40.984651 1322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:45:40.985975 1322 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:45:40.986014 1322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:45:40.986042 1322 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.194
I20260806 22:45:40.987622 1322 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:43239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab
--principal=kudu/127.18.45.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.45.194
--webserver_port=0
--tserver_master_addrs=127.18.45.254:41319
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1322
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.194
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:45:40.987929 1322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:45:40.988214 1322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:45:40.994339 1335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:40.994360 1333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:40.994323 1330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:40.994673 1322 server_base.cc:1034] running on GCE node
I20260806 22:45:40.994906 1322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:45:40.995205 1322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:45:40.996389 1322 hybrid_clock.cc:648] HybridClock initialized: now 1786056340996369 us; error 38 us; skew 500 ppm
Aug 06 22:45:40 dist-test-slave-h3ft krb5kdc[945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056340, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:45:40.999506 1322 init.cc:377] Logged in from keytab as kudu/127.18.45.194@KRBTEST.COM (short username kudu)
I20260806 22:45:41.000043 1322 webserver.cc:460] Webserver started at http://127.18.45.194:38183/ using document root <none> and password file <none>
I20260806 22:45:41.000317 1322 fs_manager.cc:359] Metadata directory not provided
I20260806 22:45:41.000370 1322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:45:41.000571 1322 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:45:41.001820 1322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/minicluster-data/ts-1/data/instance:
uuid: "2ae2c2c82ea0498eaab42d3b806ac792"
format_stamp: "Formatted at 2026-08-06 22:45:40 on dist-test-slave-h3ft"
server_key: "89f5642acce0ea7001a5879d0eca963b"
server_key_iv: "6ba8ddd41488e7271f85207aef4562d0"
server_key_version: "encryptionkey@0"
I20260806 22:45:41.002250 1322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/minicluster-data/ts-1/wal/instance:
uuid: "2ae2c2c82ea0498eaab42d3b806ac792"
format_stamp: "Formatted at 2026-08-06 22:45:40 on dist-test-slave-h3ft"
server_key: "89f5642acce0ea7001a5879d0eca963b"
server_key_iv: "6ba8ddd41488e7271f85207aef4562d0"
server_key_version: "encryptionkey@0"
I20260806 22:45:41.004082 1322 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260806 22:45:41.004988 1343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:41.005270 1322 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:45:41.005409 1322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/minicluster-data/ts-1/wal
uuid: "2ae2c2c82ea0498eaab42d3b806ac792"
format_stamp: "Formatted at 2026-08-06 22:45:40 on dist-test-slave-h3ft"
server_key: "89f5642acce0ea7001a5879d0eca963b"
server_key_iv: "6ba8ddd41488e7271f85207aef4562d0"
server_key_version: "encryptionkey@0"
I20260806 22:45:41.005501 1322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:45:41.030563 1322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:45:41.033488 1322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:45:41.033684 1322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:45:41.034044 1322 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:45:41.034482 1322 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:45:41.034523 1322 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:41.034577 1322 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:45:41.034608 1322 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:41.042645 1322 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.194:36373
I20260806 22:45:41.042734 1508 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.194:36373 every 8 connection(s)
I20260806 22:45:41.043262 1322 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
Aug 06 22:45:41 dist-test-slave-h3ft krb5kdc[945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056340, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.194@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:45:41.052548 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 1322
I20260806 22:45:41.052665 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/minicluster-data/ts-1/wal/instance
I20260806 22:45:41.052927 18615 external_mini_cluster.cc:1428] Setting key a3df4e00e6cac05a2b8fadb724e0bc11
I20260806 22:45:41.054394 1073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:41.044217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:52143 (local address 127.18.45.254:41319)
0806 22:45:41.044320 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:41.044323 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:41.044561 (+ 238us) server_negotiation.cc:407] Connection header received
0806 22:45:41.045185 (+ 624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:41.045186 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:41.045197 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:41.045271 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:41.046392 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:41.047050 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:41.048156 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:41.048389 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:41.048466 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:41.050874 (+ 2408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:41.050877 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:41.050877 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:41.050885 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:41.052731 (+ 1846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:41.053271 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:41.053273 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:41.053273 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:41.053349 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:41.053643 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:41.053644 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:41.053644 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:41.053780 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:41.053859 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:41.054178 (+ 319us) server_negotiation.cc:299] Negotiation successful
0806 22:45:41.054278 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 22:45:41.054612 1511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:41.044181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:41319 (local address 127.18.45.194:52143)
0806 22:45:41.044400 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:41.044454 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:41.045092 (+ 638us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:41.045337 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:41.045340 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:41.046020 (+ 680us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:41.046265 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:41.046270 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:41.047150 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:41.047151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:41.048052 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:41.048054 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:41.048481 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:41.048482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:41.048601 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:41.049208 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:41.049218 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:41.050772 (+ 1554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:45:41.052818 (+ 2046us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:41.052827 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:41.052830 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:41.053170 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:41.053431 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:41.053432 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:41.053432 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:41.053564 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:41.053848 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:45:41.053856 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:45:41.054026 (+ 170us) client_negotiation.cc:769] Sending connection context
0806 22:45:41.054143 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 22:45:41.054334 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":130,"threads_started":1}
I20260806 22:45:41.055636 1509 heartbeater.cc:347] Connected to a master server at 127.18.45.254:41319
I20260806 22:45:41.055768 1509 heartbeater.cc:464] Registering TS with master...
I20260806 22:45:41.056015 1509 heartbeater.cc:511] Master 127.18.45.254:41319 requested a full tablet report, sending...
I20260806 22:45:41.056564 1012 ts_manager.cc:194] Registered new tserver with Master: 2ae2c2c82ea0498eaab42d3b806ac792 (127.18.45.194:36373)
I20260806 22:45:41.056998 1012 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.45.194@KRBTEST.COM'} at 127.18.45.194:52143
WARNING: no policy specified for kudu/127.18.45.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab.
Entry for principal kudu/127.18.45.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab.
I20260806 22:45:41.086953 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.195:0
--local_ip_for_outbound_sockets=127.18.45.195
--webserver_interface=127.18.45.195
--webserver_port=0
--tserver_master_addrs=127.18.45.254:41319
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab
--principal=kudu/127.18.45.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:43239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:45:41.191608 1515 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:45:41.191870 1515 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:45:41.191900 1515 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:45:41.193189 1515 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:45:41.193236 1515 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:45:41.193265 1515 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.195
I20260806 22:45:41.194875 1515 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:43239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab
--principal=kudu/127.18.45.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.45.195
--webserver_port=0
--tserver_master_addrs=127.18.45.254:41319
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1515
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.195
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:45:41.195233 1515 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:45:41.195531 1515 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:45:41.201639 1526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:41.201643 1524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:45:41.201723 1522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:45:41.201766 1515 server_base.cc:1034] running on GCE node
I20260806 22:45:41.202131 1515 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:45:41.202512 1515 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:45:41.203693 1515 hybrid_clock.cc:648] HybridClock initialized: now 1786056341203672 us; error 36 us; skew 500 ppm
Aug 06 22:45:41 dist-test-slave-h3ft krb5kdc[945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056341, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:45:41.206823 1515 init.cc:377] Logged in from keytab as kudu/127.18.45.195@KRBTEST.COM (short username kudu)
I20260806 22:45:41.207463 1515 webserver.cc:460] Webserver started at http://127.18.45.195:37161/ using document root <none> and password file <none>
I20260806 22:45:41.207738 1515 fs_manager.cc:359] Metadata directory not provided
I20260806 22:45:41.207814 1515 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:45:41.207995 1515 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:45:41.209352 1515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/minicluster-data/ts-2/data/instance:
uuid: "0579f28b9f134849b55d9f0cb58925bb"
format_stamp: "Formatted at 2026-08-06 22:45:41 on dist-test-slave-h3ft"
server_key: "f63edac2a37c1cc3d52fd9204efd836e"
server_key_iv: "3044a8307df90cb56e879c58751b7cc8"
server_key_version: "encryptionkey@0"
I20260806 22:45:41.209784 1515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/minicluster-data/ts-2/wal/instance:
uuid: "0579f28b9f134849b55d9f0cb58925bb"
format_stamp: "Formatted at 2026-08-06 22:45:41 on dist-test-slave-h3ft"
server_key: "f63edac2a37c1cc3d52fd9204efd836e"
server_key_iv: "3044a8307df90cb56e879c58751b7cc8"
server_key_version: "encryptionkey@0"
I20260806 22:45:41.211397 1515 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260806 22:45:41.212217 1533 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:41.212404 1515 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 22:45:41.212525 1515 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/minicluster-data/ts-2/wal
uuid: "0579f28b9f134849b55d9f0cb58925bb"
format_stamp: "Formatted at 2026-08-06 22:45:41 on dist-test-slave-h3ft"
server_key: "f63edac2a37c1cc3d52fd9204efd836e"
server_key_iv: "3044a8307df90cb56e879c58751b7cc8"
server_key_version: "encryptionkey@0"
I20260806 22:45:41.212674 1515 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:45:41.224799 1515 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:45:41.227779 1515 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:45:41.228014 1515 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:45:41.228348 1515 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:45:41.228749 1515 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:45:41.228821 1515 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:41.228880 1515 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:45:41.228915 1515 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:45:41.236995 1515 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.195:34177
I20260806 22:45:41.237049 1697 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.195:34177 every 8 connection(s)
I20260806 22:45:41.237478 1515 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
Aug 06 22:45:41 dist-test-slave-h3ft krb5kdc[945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056341, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.195@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:45:41.245226 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 1515
I20260806 22:45:41.245369 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/minicluster-data/ts-2/wal/instance
I20260806 22:45:41.245617 18615 external_mini_cluster.cc:1428] Setting key dc14f0e8895636e9ff05f30a64d7a944
I20260806 22:45:41.248701 1073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:41.238375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.195:35311 (local address 127.18.45.254:41319)
0806 22:45:41.238483 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:41.238486 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:41.238701 (+ 215us) server_negotiation.cc:407] Connection header received
0806 22:45:41.239409 (+ 708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:41.239410 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:41.239425 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:41.239503 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:41.240667 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:41.241279 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:41.242422 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:41.242647 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:41.242720 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:41.245459 (+ 2739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:41.245462 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:41.245462 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:41.245475 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:41.247339 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:41.247778 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:41.247779 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:41.247779 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:41.247841 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:41.248092 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:41.248092 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:41.248093 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:41.248219 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:41.248275 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:41.248542 (+ 267us) server_negotiation.cc:299] Negotiation successful
0806 22:45:41.248619 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 22:45:41.248957 1700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:41.238328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:41319 (local address 127.18.45.195:35311)
0806 22:45:41.238556 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:41.238601 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:41.239290 (+ 689us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:41.239593 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:41.239597 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:41.240293 (+ 696us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:45:41.240540 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:41.240546 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:41.241378 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:41.241378 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:41.242330 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:41.242333 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:41.242737 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:41.242738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:41.242859 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:41.243563 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:45:41.243573 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:45:41.245335 (+ 1762us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:45:41.247421 (+ 2086us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:41.247430 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:41.247432 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:41.247706 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:41.247891 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:45:41.247892 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:45:41.247892 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:45:41.248025 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:45:41.248271 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:45:41.248275 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:45:41.248423 (+ 148us) client_negotiation.cc:769] Sending connection context
0806 22:45:41.248516 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 22:45:41.248675 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":130,"threads_started":1}
I20260806 22:45:41.249573 1698 heartbeater.cc:347] Connected to a master server at 127.18.45.254:41319
I20260806 22:45:41.249707 1698 heartbeater.cc:464] Registering TS with master...
I20260806 22:45:41.249945 1698 heartbeater.cc:511] Master 127.18.45.254:41319 requested a full tablet report, sending...
I20260806 22:45:41.250464 1012 ts_manager.cc:194] Registered new tserver with Master: 0579f28b9f134849b55d9f0cb58925bb (127.18.45.195:34177)
I20260806 22:45:41.250912 1012 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.45.195@KRBTEST.COM'} at 127.18.45.195:35311
I20260806 22:45:41.258273 18615 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:45:41.267140 1073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:41.260043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41070 (local address 127.18.45.254:41319)
0806 22:45:41.260128 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:41.260131 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:41.260231 (+ 100us) server_negotiation.cc:407] Connection header received
0806 22:45:41.260294 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:41.260295 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:41.260308 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:41.260360 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:45:41.261038 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:41.261600 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:41.262356 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:41.262559 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:41.262617 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:41.264172 (+ 1555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:45:41.264174 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:45:41.264174 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:45:41.264182 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:45:41.265771 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:41.266164 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:41.266165 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:41.266165 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:41.266225 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:45:41.266473 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:45:41.266474 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:45:41.266475 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:45:41.266646 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:45:41.266735 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:41.266897 (+ 162us) server_negotiation.cc:299] Negotiation successful
0806 22:45:41.266971 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 22:45:41.269130 1012 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41070:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:45:41.271346 1012 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:45:41.279311 1718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:41.274493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.195:34177 (local address 127.0.0.1:57118)
0806 22:45:41.275177 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:41.275184 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:41.275251 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:41.275536 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:41.275537 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:41.275539 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:45:41.275735 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:41.275737 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:41.277080 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:41.277081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:41.278157 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:41.278160 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:41.278928 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:41.278929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:41.279042 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:41.279061 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 22:45:41.279138 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 22:45:41.279206 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":668,"mutex_wait_us":3,"thread_start_us":123,"threads_started":1}
I20260806 22:45:41.279426 1719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:41.274668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44758 (local address 127.18.45.193:43103)
0806 22:45:41.274955 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:41.274962 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:41.274977 (+ 15us) server_negotiation.cc:407] Connection header received
0806 22:45:41.275188 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:41.275191 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:41.275216 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:41.275318 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:45:41.275622 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:41.276612 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:41.278224 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:41.279044 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:41.279120 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:41.279179 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:41.279264 (+ 85us) server_negotiation.cc:299] Negotiation successful
0806 22:45:41.279322 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":241,"thread_start_us":128,"threads_started":1}
I20260806 22:45:41.279629 1713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:41.274493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.194:36373 (local address 127.0.0.1:54516)
0806 22:45:41.274920 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:41.274931 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:41.275033 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:41.276017 (+ 984us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:41.276018 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:41.276020 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:45:41.276070 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:41.276071 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:41.277418 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:41.277419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:41.278312 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:41.278315 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:41.279382 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:41.279383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:41.279453 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:41.279457 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:45:41.279523 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 22:45:41.279577 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":123,"threads_started":1}
I20260806 22:45:41.279783 1721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:41.274840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57118 (local address 127.18.45.195:34177)
0806 22:45:41.275089 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:41.275093 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:41.275255 (+ 162us) server_negotiation.cc:407] Connection header received
0806 22:45:41.275334 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:41.275336 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:41.275351 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:41.275540 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:45:41.275800 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:41.276962 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:41.278265 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:41.278811 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:41.278915 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:41.278992 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:41.279638 (+ 646us) server_negotiation.cc:299] Negotiation successful
0806 22:45:41.279692 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":119,"threads_started":1}
I20260806 22:45:41.280014 1722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:41.275162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54516 (local address 127.18.45.194:36373)
0806 22:45:41.275738 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:41.275743 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:41.275755 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:45:41.275800 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:41.275802 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:41.275819 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:41.275904 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:45:41.276164 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:41.277249 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:41.278414 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:41.279229 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:41.279295 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:41.279346 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:41.279854 (+ 508us) server_negotiation.cc:299] Negotiation successful
0806 22:45:41.279911 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":524,"thread_start_us":109,"threads_started":1}
I20260806 22:45:41.280113 1714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:41.274482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.193:43103 (local address 127.0.0.1:44758)
0806 22:45:41.274835 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:41.274909 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:41.275018 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:41.275449 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:41.275453 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:41.275456 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:45:41.275551 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:41.275555 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:41.276728 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:41.276729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:41.278119 (+ 1390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:41.278122 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:41.279117 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:41.279118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:41.279191 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:41.279194 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:45:41.279247 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 22:45:41.279284 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"mutex_wait_us":31,"thread_start_us":69,"threads_started":1}
I20260806 22:45:41.280704 1600 tablet_service.cc:1467] Processing CreateTablet for tablet 7e094a5bc08048a993a21ac095a2b13a (DEFAULT_TABLE table=test-table [id=f6df7ac0e0cf4ef58be8783802c33a57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:45:41.280844 1209 tablet_service.cc:1467] Processing CreateTablet for tablet 7e094a5bc08048a993a21ac095a2b13a (DEFAULT_TABLE table=test-table [id=f6df7ac0e0cf4ef58be8783802c33a57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:45:41.281198 1600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e094a5bc08048a993a21ac095a2b13a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:45:41.281256 1209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e094a5bc08048a993a21ac095a2b13a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:45:41.283574 1418 tablet_service.cc:1467] Processing CreateTablet for tablet 7e094a5bc08048a993a21ac095a2b13a (DEFAULT_TABLE table=test-table [id=f6df7ac0e0cf4ef58be8783802c33a57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:45:41.284019 1418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e094a5bc08048a993a21ac095a2b13a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:45:41.285073 1723 tablet_bootstrap.cc:492] T 7e094a5bc08048a993a21ac095a2b13a P cef903805dbc4802bf13d0036778c46b: Bootstrap starting.
I20260806 22:45:41.285446 1724 tablet_bootstrap.cc:492] T 7e094a5bc08048a993a21ac095a2b13a P 0579f28b9f134849b55d9f0cb58925bb: Bootstrap starting.
I20260806 22:45:41.285900 1723 tablet_bootstrap.cc:654] T 7e094a5bc08048a993a21ac095a2b13a P cef903805dbc4802bf13d0036778c46b: Neither blocks nor log segments found. Creating new log.
I20260806 22:45:41.286198 1724 tablet_bootstrap.cc:654] T 7e094a5bc08048a993a21ac095a2b13a P 0579f28b9f134849b55d9f0cb58925bb: Neither blocks nor log segments found. Creating new log.
I20260806 22:45:41.286652 1723 log.cc:824] T 7e094a5bc08048a993a21ac095a2b13a P cef903805dbc4802bf13d0036778c46b: Log is configured to *not* fsync() on all Append() calls
I20260806 22:45:41.286912 1724 log.cc:824] T 7e094a5bc08048a993a21ac095a2b13a P 0579f28b9f134849b55d9f0cb58925bb: Log is configured to *not* fsync() on all Append() calls
I20260806 22:45:41.287552 1723 tablet_bootstrap.cc:492] T 7e094a5bc08048a993a21ac095a2b13a P cef903805dbc4802bf13d0036778c46b: No bootstrap required, opened a new log
I20260806 22:45:41.287642 1723 ts_tablet_manager.cc:1397] T 7e094a5bc08048a993a21ac095a2b13a P cef903805dbc4802bf13d0036778c46b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:45:41.288326 1727 tablet_bootstrap.cc:492] T 7e094a5bc08048a993a21ac095a2b13a P 2ae2c2c82ea0498eaab42d3b806ac792: Bootstrap starting.
I20260806 22:45:41.288960 1724 tablet_bootstrap.cc:492] T 7e094a5bc08048a993a21ac095a2b13a P 0579f28b9f134849b55d9f0cb58925bb: No bootstrap required, opened a new log
I20260806 22:45:41.289055 1724 ts_tablet_manager.cc:1397] T 7e094a5bc08048a993a21ac095a2b13a P 0579f28b9f134849b55d9f0cb58925bb: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 22:45:41.289083 1727 tablet_bootstrap.cc:654] T 7e094a5bc08048a993a21ac095a2b13a P 2ae2c2c82ea0498eaab42d3b806ac792: Neither blocks nor log segments found. Creating new log.
I20260806 22:45:41.289762 1727 log.cc:824] T 7e094a5bc08048a993a21ac095a2b13a P 2ae2c2c82ea0498eaab42d3b806ac792: Log is configured to *not* fsync() on all Append() calls
I20260806 22:45:41.290151 1723 raft_consensus.cc:348] T 7e094a5bc08048a993a21ac095a2b13a P cef903805dbc4802bf13d0036778c46b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0579f28b9f134849b55d9f0cb58925bb" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 34177 } } peers { permanent_uuid: "cef903805dbc4802bf13d0036778c46b" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 43103 } } peers { permanent_uuid: "2ae2c2c82ea0498eaab42d3b806ac792" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 36373 } }
I20260806 22:45:41.290338 1723 raft_consensus.cc:374] T 7e094a5bc08048a993a21ac095a2b13a P cef903805dbc4802bf13d0036778c46b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:45:41.290390 1723 raft_consensus.cc:729] T 7e094a5bc08048a993a21ac095a2b13a P cef903805dbc4802bf13d0036778c46b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cef903805dbc4802bf13d0036778c46b, State: Initialized, Role: FOLLOWER
I20260806 22:45:41.290524 1723 consensus_queue.cc:260] T 7e094a5bc08048a993a21ac095a2b13a P cef903805dbc4802bf13d0036778c46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0579f28b9f134849b55d9f0cb58925bb" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 34177 } } peers { permanent_uuid: "cef903805dbc4802bf13d0036778c46b" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 43103 } } peers { permanent_uuid: "2ae2c2c82ea0498eaab42d3b806ac792" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 36373 } }
I20260806 22:45:41.290639 1727 tablet_bootstrap.cc:492] T 7e094a5bc08048a993a21ac095a2b13a P 2ae2c2c82ea0498eaab42d3b806ac792: No bootstrap required, opened a new log
I20260806 22:45:41.290723 1727 ts_tablet_manager.cc:1397] T 7e094a5bc08048a993a21ac095a2b13a P 2ae2c2c82ea0498eaab42d3b806ac792: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:45:41.290863 1723 ts_tablet_manager.cc:1428] T 7e094a5bc08048a993a21ac095a2b13a P cef903805dbc4802bf13d0036778c46b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:45:41.291030 1314 heartbeater.cc:503] Master 127.18.45.254:41319 was elected leader, sending a full tablet report...
I20260806 22:45:41.291309 1724 raft_consensus.cc:348] T 7e094a5bc08048a993a21ac095a2b13a P 0579f28b9f134849b55d9f0cb58925bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0579f28b9f134849b55d9f0cb58925bb" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 34177 } } peers { permanent_uuid: "cef903805dbc4802bf13d0036778c46b" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 43103 } } peers { permanent_uuid: "2ae2c2c82ea0498eaab42d3b806ac792" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 36373 } }
I20260806 22:45:41.291978 1724 raft_consensus.cc:374] T 7e094a5bc08048a993a21ac095a2b13a P 0579f28b9f134849b55d9f0cb58925bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:45:41.292052 1724 raft_consensus.cc:729] T 7e094a5bc08048a993a21ac095a2b13a P 0579f28b9f134849b55d9f0cb58925bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0579f28b9f134849b55d9f0cb58925bb, State: Initialized, Role: FOLLOWER
I20260806 22:45:41.292205 1724 consensus_queue.cc:260] T 7e094a5bc08048a993a21ac095a2b13a P 0579f28b9f134849b55d9f0cb58925bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0579f28b9f134849b55d9f0cb58925bb" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 34177 } } peers { permanent_uuid: "cef903805dbc4802bf13d0036778c46b" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 43103 } } peers { permanent_uuid: "2ae2c2c82ea0498eaab42d3b806ac792" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 36373 } }
I20260806 22:45:41.292554 1724 ts_tablet_manager.cc:1428] T 7e094a5bc08048a993a21ac095a2b13a P 0579f28b9f134849b55d9f0cb58925bb: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:45:41.292752 1698 heartbeater.cc:503] Master 127.18.45.254:41319 was elected leader, sending a full tablet report...
I20260806 22:45:41.293320 1727 raft_consensus.cc:348] T 7e094a5bc08048a993a21ac095a2b13a P 2ae2c2c82ea0498eaab42d3b806ac792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0579f28b9f134849b55d9f0cb58925bb" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 34177 } } peers { permanent_uuid: "cef903805dbc4802bf13d0036778c46b" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 43103 } } peers { permanent_uuid: "2ae2c2c82ea0498eaab42d3b806ac792" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 36373 } }
I20260806 22:45:41.293499 1727 raft_consensus.cc:374] T 7e094a5bc08048a993a21ac095a2b13a P 2ae2c2c82ea0498eaab42d3b806ac792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:45:41.293538 1727 raft_consensus.cc:729] T 7e094a5bc08048a993a21ac095a2b13a P 2ae2c2c82ea0498eaab42d3b806ac792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ae2c2c82ea0498eaab42d3b806ac792, State: Initialized, Role: FOLLOWER
I20260806 22:45:41.293656 1727 consensus_queue.cc:260] T 7e094a5bc08048a993a21ac095a2b13a P 2ae2c2c82ea0498eaab42d3b806ac792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0579f28b9f134849b55d9f0cb58925bb" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 34177 } } peers { permanent_uuid: "cef903805dbc4802bf13d0036778c46b" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 43103 } } peers { permanent_uuid: "2ae2c2c82ea0498eaab42d3b806ac792" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 36373 } }
I20260806 22:45:41.293978 1727 ts_tablet_manager.cc:1428] T 7e094a5bc08048a993a21ac095a2b13a P 2ae2c2c82ea0498eaab42d3b806ac792: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:45:41.294003 1509 heartbeater.cc:503] Master 127.18.45.254:41319 was elected leader, sending a full tablet report...
W20260806 22:45:41.334501 1316 tablet.cc:2369] T 7e094a5bc08048a993a21ac095a2b13a P cef903805dbc4802bf13d0036778c46b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:45:41.348953 1732 raft_consensus.cc:482] T 7e094a5bc08048a993a21ac095a2b13a P 0579f28b9f134849b55d9f0cb58925bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:45:41.349164 1732 raft_consensus.cc:504] T 7e094a5bc08048a993a21ac095a2b13a P 0579f28b9f134849b55d9f0cb58925bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0579f28b9f134849b55d9f0cb58925bb" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 34177 } } peers { permanent_uuid: "cef903805dbc4802bf13d0036778c46b" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 43103 } } peers { permanent_uuid: "2ae2c2c82ea0498eaab42d3b806ac792" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 36373 } }
I20260806 22:45:41.349792 1732 leader_election.cc:288] T 7e094a5bc08048a993a21ac095a2b13a P 0579f28b9f134849b55d9f0cb58925bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cef903805dbc4802bf13d0036778c46b (127.18.45.193:43103), 2ae2c2c82ea0498eaab42d3b806ac792 (127.18.45.194:36373)
I20260806 22:45:41.353241 1700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:41.349983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.194:36373 (local address 127.18.45.195:39419)
0806 22:45:41.350052 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:41.350067 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:41.350118 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:41.350328 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:41.350329 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:41.350331 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:45:41.350446 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:41.350449 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:41.351417 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:41.351419 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:41.352272 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:41.352275 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:41.352957 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:41.352959 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:41.353058 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:41.353063 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:45:41.353120 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 22:45:41.353172 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260806 22:45:41.353329 1722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:41.350092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.195:39419 (local address 127.18.45.194:36373)
0806 22:45:41.350157 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:41.350160 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:41.350174 (+ 14us) server_negotiation.cc:407] Connection header received
0806 22:45:41.350212 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:41.350213 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:41.350230 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:41.350311 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:45:41.350578 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:41.351292 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:41.352386 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:41.352872 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:41.352940 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:41.352950 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:41.353166 (+ 216us) server_negotiation.cc:299] Negotiation successful
0806 22:45:41.353219 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 22:45:41.353454 1737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:41.350035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.193:43103 (local address 127.18.45.195:49609)
0806 22:45:41.350317 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:41.350331 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:45:41.350401 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:45:41.350577 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:45:41.350578 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:45:41.350580 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:45:41.350672 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:41.350674 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:41.351421 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:41.351422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:41.352344 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:45:41.352348 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:45:41.353145 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:45:41.353146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:45:41.353239 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:41.353244 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:45:41.353314 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 22:45:41.353360 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"mutex_wait_us":43,"thread_start_us":182,"threads_started":1}
I20260806 22:45:41.353776 1719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:41.350095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.195:49609 (local address 127.18.45.193:43103)
0806 22:45:41.350162 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:41.350165 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:41.350396 (+ 231us) server_negotiation.cc:407] Connection header received
0806 22:45:41.350469 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:41.350471 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:41.350490 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:41.350552 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:45:41.350755 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:41.351322 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:41.352501 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:41.352986 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:41.353064 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:41.353074 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:41.353670 (+ 596us) server_negotiation.cc:299] Negotiation successful
0806 22:45:41.353716 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 22:45:41.353765 1443 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e094a5bc08048a993a21ac095a2b13a" candidate_uuid: "0579f28b9f134849b55d9f0cb58925bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ae2c2c82ea0498eaab42d3b806ac792" is_pre_election: true
I20260806 22:45:41.354017 1443 raft_consensus.cc:2454] T 7e094a5bc08048a993a21ac095a2b13a P 2ae2c2c82ea0498eaab42d3b806ac792 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0579f28b9f134849b55d9f0cb58925bb in term 0.
I20260806 22:45:41.354223 1240 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e094a5bc08048a993a21ac095a2b13a" candidate_uuid: "0579f28b9f134849b55d9f0cb58925bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cef903805dbc4802bf13d0036778c46b" is_pre_election: true
I20260806 22:45:41.354323 1534 leader_election.cc:302] T 7e094a5bc08048a993a21ac095a2b13a P 0579f28b9f134849b55d9f0cb58925bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0579f28b9f134849b55d9f0cb58925bb, 2ae2c2c82ea0498eaab42d3b806ac792; no voters:
I20260806 22:45:41.354452 1240 raft_consensus.cc:2454] T 7e094a5bc08048a993a21ac095a2b13a P cef903805dbc4802bf13d0036778c46b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0579f28b9f134849b55d9f0cb58925bb in term 0.
I20260806 22:45:41.354465 1732 raft_consensus.cc:2781] T 7e094a5bc08048a993a21ac095a2b13a P 0579f28b9f134849b55d9f0cb58925bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:45:41.354537 1732 raft_consensus.cc:482] T 7e094a5bc08048a993a21ac095a2b13a P 0579f28b9f134849b55d9f0cb58925bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:45:41.354570 1732 raft_consensus.cc:3037] T 7e094a5bc08048a993a21ac095a2b13a P 0579f28b9f134849b55d9f0cb58925bb [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:45:41.355589 1732 raft_consensus.cc:504] T 7e094a5bc08048a993a21ac095a2b13a P 0579f28b9f134849b55d9f0cb58925bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0579f28b9f134849b55d9f0cb58925bb" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 34177 } } peers { permanent_uuid: "cef903805dbc4802bf13d0036778c46b" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 43103 } } peers { permanent_uuid: "2ae2c2c82ea0498eaab42d3b806ac792" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 36373 } }
I20260806 22:45:41.355762 1732 leader_election.cc:288] T 7e094a5bc08048a993a21ac095a2b13a P 0579f28b9f134849b55d9f0cb58925bb [CANDIDATE]: Term 1 election: Requested vote from peers cef903805dbc4802bf13d0036778c46b (127.18.45.193:43103), 2ae2c2c82ea0498eaab42d3b806ac792 (127.18.45.194:36373)
I20260806 22:45:41.355948 1240 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e094a5bc08048a993a21ac095a2b13a" candidate_uuid: "0579f28b9f134849b55d9f0cb58925bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cef903805dbc4802bf13d0036778c46b"
I20260806 22:45:41.355994 1443 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e094a5bc08048a993a21ac095a2b13a" candidate_uuid: "0579f28b9f134849b55d9f0cb58925bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ae2c2c82ea0498eaab42d3b806ac792"
I20260806 22:45:41.356062 1240 raft_consensus.cc:3037] T 7e094a5bc08048a993a21ac095a2b13a P cef903805dbc4802bf13d0036778c46b [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:45:41.356107 1443 raft_consensus.cc:3037] T 7e094a5bc08048a993a21ac095a2b13a P 2ae2c2c82ea0498eaab42d3b806ac792 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:45:41.356982 1240 raft_consensus.cc:2454] T 7e094a5bc08048a993a21ac095a2b13a P cef903805dbc4802bf13d0036778c46b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0579f28b9f134849b55d9f0cb58925bb in term 1.
I20260806 22:45:41.357118 1443 raft_consensus.cc:2454] T 7e094a5bc08048a993a21ac095a2b13a P 2ae2c2c82ea0498eaab42d3b806ac792 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0579f28b9f134849b55d9f0cb58925bb in term 1.
I20260806 22:45:41.357182 1537 leader_election.cc:302] T 7e094a5bc08048a993a21ac095a2b13a P 0579f28b9f134849b55d9f0cb58925bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0579f28b9f134849b55d9f0cb58925bb, cef903805dbc4802bf13d0036778c46b; no voters:
I20260806 22:45:41.357323 1732 raft_consensus.cc:2781] T 7e094a5bc08048a993a21ac095a2b13a P 0579f28b9f134849b55d9f0cb58925bb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:45:41.357506 1732 raft_consensus.cc:686] T 7e094a5bc08048a993a21ac095a2b13a P 0579f28b9f134849b55d9f0cb58925bb [term 1 LEADER]: Becoming Leader. State: Replica: 0579f28b9f134849b55d9f0cb58925bb, State: Running, Role: LEADER
I20260806 22:45:41.357633 1732 consensus_queue.cc:237] T 7e094a5bc08048a993a21ac095a2b13a P 0579f28b9f134849b55d9f0cb58925bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0579f28b9f134849b55d9f0cb58925bb" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 34177 } } peers { permanent_uuid: "cef903805dbc4802bf13d0036778c46b" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 43103 } } peers { permanent_uuid: "2ae2c2c82ea0498eaab42d3b806ac792" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 36373 } }
I20260806 22:45:41.358899 1011 catalog_manager.cc:5495] T 7e094a5bc08048a993a21ac095a2b13a P 0579f28b9f134849b55d9f0cb58925bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 0579f28b9f134849b55d9f0cb58925bb (127.18.45.195). New cstate: current_term: 1 leader_uuid: "0579f28b9f134849b55d9f0cb58925bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0579f28b9f134849b55d9f0cb58925bb" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 34177 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cef903805dbc4802bf13d0036778c46b" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 43103 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ae2c2c82ea0498eaab42d3b806ac792" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 36373 } health_report { overall_health: UNKNOWN } } }
I20260806 22:45:41.374636 1721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:41.371786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57128 (local address 127.18.45.195:34177)
0806 22:45:41.371868 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0806 22:45:41.371870 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:45:41.371881 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:45:41.371942 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:45:41.371943 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:45:41.371970 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:45:41.372059 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:45:41.372244 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:41.372798 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:41.373679 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:45:41.373886 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:45:41.373967 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:45:41.374185 (+ 218us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:45:41.374327 (+ 142us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:45:41.374437 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 22:45:41.374458 (+ 21us) server_negotiation.cc:299] Negotiation successful
0806 22:45:41.374486 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 22:45:41.377274 1240 raft_consensus.cc:1261] T 7e094a5bc08048a993a21ac095a2b13a P cef903805dbc4802bf13d0036778c46b [term 1 FOLLOWER]: Refusing update from remote peer 0579f28b9f134849b55d9f0cb58925bb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:45:41.377274 1443 raft_consensus.cc:1261] T 7e094a5bc08048a993a21ac095a2b13a P 2ae2c2c82ea0498eaab42d3b806ac792 [term 1 FOLLOWER]: Refusing update from remote peer 0579f28b9f134849b55d9f0cb58925bb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:45:41.377734 1732 consensus_queue.cc:1035] T 7e094a5bc08048a993a21ac095a2b13a P 0579f28b9f134849b55d9f0cb58925bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ae2c2c82ea0498eaab42d3b806ac792" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 36373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:45:41.378046 1738 consensus_queue.cc:1035] T 7e094a5bc08048a993a21ac095a2b13a P 0579f28b9f134849b55d9f0cb58925bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "cef903805dbc4802bf13d0036778c46b" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 43103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:45:41.381001 1743 mvcc.cc:205] Tried to move back new op lower bound from 7315686774277648384 to 7315686774201974784. Current Snapshot: MvccSnapshot[applied={T|T < 7315686774277648384}]
I20260806 22:45:41.381292 1747 mvcc.cc:205] Tried to move back new op lower bound from 7315686774277648384 to 7315686774201974784. Current Snapshot: MvccSnapshot[applied={T|T < 7315686774277648384}]
I20260806 22:45:41.382128 1744 mvcc.cc:205] Tried to move back new op lower bound from 7315686774277648384 to 7315686774201974784. Current Snapshot: MvccSnapshot[applied={T|T < 7315686774277648384 or (T in {7315686774277648384})}]
I20260806 22:45:41.384335 1012 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41070:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:45:41.384476 1012 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41070:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:45:41.385533 1012 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee605b5e78f54a7584404506992c376e: Sending DeleteTablet for 3 replicas of tablet 7e094a5bc08048a993a21ac095a2b13a
I20260806 22:45:41.385947 1600 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e094a5bc08048a993a21ac095a2b13a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:45:41 UTC) from {username='kudu', principal='kudu/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:57118
I20260806 22:45:41.386008 1209 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e094a5bc08048a993a21ac095a2b13a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:45:41 UTC) from {username='kudu', principal='kudu/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:44758
I20260806 22:45:41.386307 1757 tablet_replica.cc:330] T 7e094a5bc08048a993a21ac095a2b13a P cef903805dbc4802bf13d0036778c46b: stopping tablet replica
I20260806 22:45:41.386338 1756 tablet_replica.cc:330] T 7e094a5bc08048a993a21ac095a2b13a P 0579f28b9f134849b55d9f0cb58925bb: stopping tablet replica
I20260806 22:45:41.386464 1756 raft_consensus.cc:2229] T 7e094a5bc08048a993a21ac095a2b13a P 0579f28b9f134849b55d9f0cb58925bb [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:45:41.386484 1418 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e094a5bc08048a993a21ac095a2b13a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:45:41 UTC) from {username='kudu', principal='kudu/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:54516
I20260806 22:45:41.386626 1756 raft_consensus.cc:2258] T 7e094a5bc08048a993a21ac095a2b13a P 0579f28b9f134849b55d9f0cb58925bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:45:41.386562 1757 raft_consensus.cc:2229] T 7e094a5bc08048a993a21ac095a2b13a P cef903805dbc4802bf13d0036778c46b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:45:41.386824 1757 raft_consensus.cc:2258] T 7e094a5bc08048a993a21ac095a2b13a P cef903805dbc4802bf13d0036778c46b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:45:41.387279 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 1113
I20260806 22:45:41.387528 1757 ts_tablet_manager.cc:1905] T 7e094a5bc08048a993a21ac095a2b13a P cef903805dbc4802bf13d0036778c46b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:45:41.395402 1758 tablet_replica.cc:330] T 7e094a5bc08048a993a21ac095a2b13a P 2ae2c2c82ea0498eaab42d3b806ac792: stopping tablet replica
I20260806 22:45:41.395743 1758 raft_consensus.cc:2229] T 7e094a5bc08048a993a21ac095a2b13a P 2ae2c2c82ea0498eaab42d3b806ac792 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:45:41.395987 1758 raft_consensus.cc:2258] T 7e094a5bc08048a993a21ac095a2b13a P 2ae2c2c82ea0498eaab42d3b806ac792 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:45:41.396813 1756 ts_tablet_manager.cc:1905] T 7e094a5bc08048a993a21ac095a2b13a P 0579f28b9f134849b55d9f0cb58925bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:45:41.399534 1756 ts_tablet_manager.cc:1918] T 7e094a5bc08048a993a21ac095a2b13a P 0579f28b9f134849b55d9f0cb58925bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:45:41.399652 1756 log.cc:1196] T 7e094a5bc08048a993a21ac095a2b13a P 0579f28b9f134849b55d9f0cb58925bb: Deleting WAL directory at /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/minicluster-data/ts-2/wal/wals/7e094a5bc08048a993a21ac095a2b13a
I20260806 22:45:41.400048 1756 ts_tablet_manager.cc:1939] T 7e094a5bc08048a993a21ac095a2b13a P 0579f28b9f134849b55d9f0cb58925bb: Deleting consensus metadata
I20260806 22:45:41.400686 991 catalog_manager.cc:4841] TS 0579f28b9f134849b55d9f0cb58925bb (127.18.45.195:34177): tablet 7e094a5bc08048a993a21ac095a2b13a (table test-table [id=f6df7ac0e0cf4ef58be8783802c33a57]) successfully deleted
I20260806 22:45:41.401072 1758 ts_tablet_manager.cc:1905] T 7e094a5bc08048a993a21ac095a2b13a P 2ae2c2c82ea0498eaab42d3b806ac792: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:45:41.402658 1758 ts_tablet_manager.cc:1918] T 7e094a5bc08048a993a21ac095a2b13a P 2ae2c2c82ea0498eaab42d3b806ac792: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:45:41.402769 1758 log.cc:1196] T 7e094a5bc08048a993a21ac095a2b13a P 2ae2c2c82ea0498eaab42d3b806ac792: Deleting WAL directory at /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786056275803658-18615-0/minicluster-data/ts-1/wal/wals/7e094a5bc08048a993a21ac095a2b13a
I20260806 22:45:41.403121 1758 ts_tablet_manager.cc:1939] T 7e094a5bc08048a993a21ac095a2b13a P 2ae2c2c82ea0498eaab42d3b806ac792: Deleting consensus metadata
W20260806 22:45:41.403407 993 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.45.193:43103 (error 108)
I20260806 22:45:41.403718 989 catalog_manager.cc:4841] TS 2ae2c2c82ea0498eaab42d3b806ac792 (127.18.45.194:36373): tablet 7e094a5bc08048a993a21ac095a2b13a (table test-table [id=f6df7ac0e0cf4ef58be8783802c33a57]) successfully deleted
I20260806 22:45:41.404114 1718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:45:41.403963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.193:43103 (local address 127.0.0.1:44766)
0806 22:45:41.404030 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0806 22:45:41.404077 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.193:43103: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260806 22:45:41.404248 993 catalog_manager.cc:4584] TS cef903805dbc4802bf13d0036778c46b (127.18.45.193:43103): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7e094a5bc08048a993a21ac095a2b13a: Network error: Client connection negotiation failed: client connection to 127.18.45.193:43103: connect: Connection refused (error 111)
I20260806 22:45:41.404626 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 1322
I20260806 22:45:41.422784 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 1515
I20260806 22:45:41.437273 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 964
2026-08-06T22:45:41Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2450 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:45:41 dist-test-slave-h3ft krb5kdc[1762](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:45:41 dist-test-slave-h3ft krb5kdc[1762](info): set up 2 sockets
Aug 06 22:45:41 dist-test-slave-h3ft krb5kdc[1762](info): commencing operation
krb5kdc: starting...
W20260806 22:45:42.717747 18615 mini_kdc.cc:121] Time spent starting KDC: real 1.252s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Aug 06 22:45:42 dist-test-slave-h3ft krb5kdc[1762](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056342, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:45:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:45:42Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.18.45.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.18.45.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.18.45.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/krb5kdc/rangeradmin_127.18.45.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.18.45.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/krb5kdc/rangeradmin_127.18.45.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.18.45.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.18.45.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.18.45.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/krb5kdc/rangerlookup_127.18.45.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.18.45.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/krb5kdc/rangerlookup_127.18.45.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.18.45.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.18.45.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.18.45.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/krb5kdc/HTTP_127.18.45.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.18.45.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/krb5kdc/HTTP_127.18.45.212@KRBTEST.COM.keytab.
I20260806 22:45:42.931268 18615 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskXSWSdD/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/postgres -l logfile start
2026-08-06 22:45:45.854 UTC [1807] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 22:45:45.854 UTC [1807] LOG: listening on IPv4 address "127.18.45.212", port 34447
2026-08-06 22:45:45.858 UTC [1807] LOG: listening on Unix socket "/tmp/.s.PGSQL.34447"
2026-08-06 22:45:45.886 UTC [1810] LOG: database system was shut down at 2026-08-06 22:45:44 UTC
2026-08-06 22:45:45.890 UTC [1807] LOG: database system is ready to accept connections
I20260806 22:45:47.038493 18615 mini_postgres.cc:96] Postgres bound to 34447
2026-08-06 22:45:47.047 UTC [1822] FATAL: database "slave" does not exist
127.18.45.212:34447 - accepting connections
I20260806 22:45:47.048534 18615 mini_ranger.cc:161] Starting Ranger...
I20260806 22:45:47.066746 18615 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 22:45:47.638756 18615 mini_ranger.cc:87] Created ranger Postgres database
I20260806 22:45:47.638912 18615 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/ranger-admin
2026-08-06 22:45:47,953 [I] DB FLAVOR :POSTGRES
2026-08-06 22:45:47,953 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 22:45:47,953 [I] Checking connection..
2026-08-06 22:45:47,954 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXSWSdD/build/release/bin/postgresql.jar:/tmp/dist-test-taskXSWSdD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.45.212:34447/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 22:45:48,263 [I] Checking connection passed.
2026-08-06 22:45:48,263 [I] --------- Verifying version history table ---------
2026-08-06 22:45:48,263 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXSWSdD/build/release/bin/postgresql.jar:/tmp/dist-test-taskXSWSdD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.45.212:34447/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 22:45:48,563 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 22:45:48,563 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXSWSdD/build/release/bin/postgresql.jar:/tmp/dist-test-taskXSWSdD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.45.212:34447/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 22:45:48,857 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 22:45:48,858 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 22:45:48,858 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXSWSdD/build/release/bin/postgresql.jar:/tmp/dist-test-taskXSWSdD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.45.212:34447/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskXSWSdD/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 22:45:49.166 UTC [1913] WARNING: there is no transaction in progress
2026-08-06 22:45:49,176 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 22:45:49,177 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXSWSdD/build/release/bin/postgresql.jar:/tmp/dist-test-taskXSWSdD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.45.212:34447/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 22:45:49,471 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 22:45:49,472 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 22:45:49,472 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXSWSdD/build/release/bin/postgresql.jar:/tmp/dist-test-taskXSWSdD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.45.212:34447/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 22:45:49,756 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXSWSdD/build/release/bin/postgresql.jar:/tmp/dist-test-taskXSWSdD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.45.212:34447/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 22:45:50,072 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXSWSdD/build/release/bin/postgresql.jar:/tmp/dist-test-taskXSWSdD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.45.212:34447/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-h3ft.c.gcp-upstream.internal','N') ;"
2026-08-06 22:45:50,367 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXSWSdD/build/release/bin/postgresql.jar:/tmp/dist-test-taskXSWSdD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.45.212:34447/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 22:45:50,674 [I] Table x_portal_user does not exist in database ranger
2026-08-06 22:45:50,674 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXSWSdD/build/release/bin/postgresql.jar:/tmp/dist-test-taskXSWSdD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.45.212:34447/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 22:45:50,992 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 22:45:50,992 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXSWSdD/build/release/bin/postgresql.jar:/tmp/dist-test-taskXSWSdD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.45.212:34447/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 22:45:51,287 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXSWSdD/build/release/bin/postgresql.jar:/tmp/dist-test-taskXSWSdD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.45.212:34447/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 22:45:51,607 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 22:45:51,608 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXSWSdD/build/release/bin/postgresql.jar:/tmp/dist-test-taskXSWSdD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.45.212:34447/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskXSWSdD/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 22:45:52.770 UTC [2085] WARNING: there is no transaction in progress
2026-08-06 22:45:52.788 UTC [2085] WARNING: there is no transaction in progress
2026-08-06 22:45:52.807 UTC [2085] WARNING: there is no transaction in progress
2026-08-06 22:45:52.826 UTC [2085] WARNING: there is no transaction in progress
2026-08-06 22:45:52.846 UTC [2085] WARNING: there is no transaction in progress
2026-08-06 22:45:52.869 UTC [2085] WARNING: there is no transaction in progress
2026-08-06 22:45:52.935 UTC [2085] WARNING: there is no transaction in progress
2026-08-06 22:45:52.943 UTC [2085] WARNING: there is no transaction in progress
2026-08-06 22:45:52.960 UTC [2085] WARNING: there is no transaction in progress
2026-08-06 22:45:52.972 UTC [2085] WARNING: there is no transaction in progress
2026-08-06 22:45:52.984 UTC [2085] WARNING: there is no transaction in progress
2026-08-06 22:45:53.002 UTC [2085] WARNING: there is no transaction in progress
2026-08-06 22:45:53.012 UTC [2085] WARNING: there is no transaction in progress
2026-08-06 22:45:53.020 UTC [2085] WARNING: there is no transaction in progress
2026-08-06 22:45:53.028 UTC [2085] WARNING: there is no transaction in progress
2026-08-06 22:45:53.036 UTC [2085] WARNING: there is no transaction in progress
2026-08-06 22:45:53,829 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 22:45:53,830 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXSWSdD/build/release/bin/postgresql.jar:/tmp/dist-test-taskXSWSdD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.45.212:34447/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 22:45:54,113 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 22:45:54,113 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXSWSdD/build/release/bin/postgresql.jar:/tmp/dist-test-taskXSWSdD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.45.212:34447/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 22:45:54,421 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 22:45:54,422 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXSWSdD/build/release/bin/postgresql.jar:/tmp/dist-test-taskXSWSdD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.45.212:34447/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 22:45:54,728 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 22:45:54,728 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXSWSdD/build/release/bin/postgresql.jar:/tmp/dist-test-taskXSWSdD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.45.212:34447/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 22:45:55,018 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXSWSdD/build/release/bin/postgresql.jar:/tmp/dist-test-taskXSWSdD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.45.212:34447/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 22:45:55,304 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXSWSdD/build/release/bin/postgresql.jar:/tmp/dist-test-taskXSWSdD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.45.212:34447/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-h3ft.c.gcp-upstream.internal';"
2026-08-06 22:45:55,592 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 22:45:55,593 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXSWSdD/build/release/bin/postgresql.jar:/tmp/dist-test-taskXSWSdD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.45.212:34447/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 22:45:55,910 [I] DB_PATCHES have already been applied
I20260806 22:45:55.919435 18615 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/ranger-admin:/tmp/dist-test-taskXSWSdD/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskXSWSdD/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskXSWSdD/build/release/bin/postgresql.jar:/tmp/dist-test-taskXSWSdD/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 22:45:55.919545 18615 mini_ranger.cc:193] Using host: 127.18.45.212
I20260806 22:45:55.924979 18615 mini_ranger.cc:227] Ranger admin URL: http://127.18.45.212:58761
Aug 06, 2026 10:45:56 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 10:45:56 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 10:45:56 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 10:45:56 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 10:45:56 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 10:45:57 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 10:45:57 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 22:45:57.333633 18615 mini_ranger.cc:160] Time spent starting Ranger: real 10.285s user 0.000s sys 0.015s
Aug 06, 2026 10:46:21 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 10:46:21 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 06, 2026 10:46:21 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 10:46:21 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 10:46:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 10:46:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 10:46:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 10:46:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 10:46:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 10:46:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 10:46:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 10:46:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 10:46:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 10:46:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 10:46:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 10:46:21 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 10:46:21 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 10:46:22 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 22:46:23.161985 18615 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.18.45.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.18.45.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.18.45.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/krb5kdc/rangerkms_127.18.45.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.18.45.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/krb5kdc/rangerkms_127.18.45.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.18.45.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.18.45.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.18.45.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/krb5kdc/HTTP_127.18.45.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.18.45.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/krb5kdc/HTTP_127.18.45.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 22:46:23 dist-test-slave-h3ft krb5kdc[1762](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056383, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 22:46:23.285105 18615 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 22:46:23.302356 18615 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 22:46:24.349419 18615 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 22:46:24.349571 18615 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/ranger-kms
I20260806 22:46:24.349601 18615 mini_ranger_kms.cc:227] Using postgres at 127.18.45.212:34447
2026-08-06 22:46:24,637 [I] DB FLAVOR :POSTGRES
2026-08-06 22:46:24,637 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 22:46:24,637 [I] Checking connection
2026-08-06 22:46:24,637 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXSWSdD/build/release/bin/postgresql.jar:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.45.212:34447/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 22:46:24,923 [I] connection success
2026-08-06 22:46:24,924 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 22:46:24,924 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 22:46:24,924 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXSWSdD/build/release/bin/postgresql.jar:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.45.212:34447/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 22:46:25,199 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 22:46:25,200 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 22:46:25,200 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 22:46:25,200 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXSWSdD/build/release/bin/postgresql.jar:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.45.212:34447/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 22:46:25,488 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 22:46:26.145255 18615 mini_ranger_kms.cc:310] Created kms service
I20260806 22:46:26.393216 18615 mini_ranger_kms.cc:326] Created kudu user
I20260806 22:46:26.444766 18615 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 22:46:26 dist-test-slave-h3ft krb5kdc[1762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056383, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.18.45.212@KRBTEST.COM
I20260806 22:46:26.957613 18615 mini_ranger_kms.cc:384] Added ranger policy
I20260806 22:46:26.957798 18615 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/ranger-kms:/tmp/dist-test-taskXSWSdD/build/release/bin/postgresql.jar:/tmp/dist-test-taskXSWSdD/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskXSWSdD/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 22:46:26.957859 18615 mini_ranger_kms.cc:240] Using host: 127.18.45.212
I20260806 22:46:26.964380 18615 mini_ranger_kms.cc:276] Ranger KMS PID: 2365
I20260806 22:46:26.964601 18615 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.18.45.212:55641
Aug 06, 2026 10:46:28 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 10:46:28 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 10:46:28 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 10:46:28 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 10:46:28 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 10:46:28 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260806 22:46:29.751765 18615 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.467s user 0.000s sys 0.272s
I20260806 22:46:29.751941 18615 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 22:46:29.752090 18615 mini_ranger_kms.cc:401] 127.18.45.212:55641/kms/v1/keys
Aug 06 22:46:29 dist-test-slave-h3ft krb5kdc[1762](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786056389, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.45.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 06 22:46:30 dist-test-slave-h3ft krb5kdc[1762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056389, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.45.212@KRBTEST.COM for HTTP/127.18.45.212@KRBTEST.COM
Aug 06 22:46:30 dist-test-slave-h3ft krb5kdc[1762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056389, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.45.212@KRBTEST.COM for HTTP/127.18.45.212@KRBTEST.COM
Aug 06 22:46:33 dist-test-slave-h3ft krb5kdc[1762](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056393, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.18.45.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab.
Entry for principal kudu/127.18.45.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab.
I20260806 22:46:33.538740 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:33077
--webserver_interface=127.18.45.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:32783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.45.254:33077
--ranger_config_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.45.212:55641/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:46:33.648965 2440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:46:33.649286 2440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:46:33.649313 2440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:46:33.650735 2440 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:46:33.650790 2440 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:46:33.650805 2440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:46:33.650818 2440 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:46:33.650830 2440 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:46:33.652946 2440 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:32783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/master-0/wal
--ranger_kms_url=127.18.45.212:55641/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.18.45.254:33077
--ranger_config_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:33077
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.45.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2440
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:46:33.653276 2440 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:46:33.653575 2440 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 22:46:33.660051 2440 server_base.cc:1034] running on GCE node
W20260806 22:46:33.660059 2449 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:33.660284 2448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:33.660384 2452 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:33.660724 2440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:46:33.661109 2440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:46:33.662320 2440 hybrid_clock.cc:648] HybridClock initialized: now 1786056393662295 us; error 50 us; skew 500 ppm
Aug 06 22:46:33 dist-test-slave-h3ft krb5kdc[1762](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056393, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:46:33.665810 2440 init.cc:377] Logged in from keytab as kudu/127.18.45.254@KRBTEST.COM (short username kudu)
I20260806 22:46:33.666416 2440 webserver.cc:460] Webserver started at http://127.18.45.254:35153/ using document root <none> and password file <none>
I20260806 22:46:33.666687 2440 fs_manager.cc:359] Metadata directory not provided
I20260806 22:46:33.666755 2440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:46:33.666931 2440 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 22:46:33 dist-test-slave-h3ft krb5kdc[1762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056393, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.254@KRBTEST.COM for HTTP/127.18.45.212@KRBTEST.COM
I20260806 22:46:33.742060 2440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/master-0/data/instance:
uuid: "d5736fe9e8c04cb5b5b25d5a92a6914f"
format_stamp: "Formatted at 2026-08-06 22:46:33 on dist-test-slave-h3ft"
server_key: "90dc1632c9c3d407f9e52239a2000592"
server_key_iv: "76b7fca3beff574ba7caf632c81c0017"
server_key_version: "kuduclusterkey@0"
I20260806 22:46:33.742623 2440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/master-0/wal/instance:
uuid: "d5736fe9e8c04cb5b5b25d5a92a6914f"
format_stamp: "Formatted at 2026-08-06 22:46:33 on dist-test-slave-h3ft"
server_key: "90dc1632c9c3d407f9e52239a2000592"
server_key_iv: "76b7fca3beff574ba7caf632c81c0017"
server_key_version: "kuduclusterkey@0"
I20260806 22:46:33.744742 2440 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260806 22:46:33.763943 2461 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:33.764197 2440 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:46:33.764328 2440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/master-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/master-0/wal
uuid: "d5736fe9e8c04cb5b5b25d5a92a6914f"
format_stamp: "Formatted at 2026-08-06 22:46:33 on dist-test-slave-h3ft"
server_key: "90dc1632c9c3d407f9e52239a2000592"
server_key_iv: "76b7fca3beff574ba7caf632c81c0017"
server_key_version: "kuduclusterkey@0"
I20260806 22:46:33.764479 2440 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:46:33.785668 2440 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:46:33.788292 2440 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:46:33.788498 2440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:46:33.795432 2440 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.254:33077
I20260806 22:46:33.795475 2545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.254:33077 every 8 connection(s)
I20260806 22:46:33.795998 2440 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
I20260806 22:46:33.796991 2547 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:33.799274 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 2440
I20260806 22:46:33.799425 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/master-0/wal/instance
I20260806 22:46:33.799685 18615 external_mini_cluster.cc:1428] Setting key baf63c18e3e9fe2dd3cf0813882a2fb8
I20260806 22:46:33.800752 2547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5736fe9e8c04cb5b5b25d5a92a6914f: Bootstrap starting.
I20260806 22:46:33.801911 2547 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d5736fe9e8c04cb5b5b25d5a92a6914f: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:33.802450 2547 log.cc:824] T 00000000000000000000000000000000 P d5736fe9e8c04cb5b5b25d5a92a6914f: Log is configured to *not* fsync() on all Append() calls
I20260806 22:46:33.803499 2547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5736fe9e8c04cb5b5b25d5a92a6914f: No bootstrap required, opened a new log
I20260806 22:46:33.806727 2547 raft_consensus.cc:348] T 00000000000000000000000000000000 P d5736fe9e8c04cb5b5b25d5a92a6914f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5736fe9e8c04cb5b5b25d5a92a6914f" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 33077 } }
I20260806 22:46:33.806983 2547 raft_consensus.cc:374] T 00000000000000000000000000000000 P d5736fe9e8c04cb5b5b25d5a92a6914f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:33.807061 2547 raft_consensus.cc:729] T 00000000000000000000000000000000 P d5736fe9e8c04cb5b5b25d5a92a6914f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5736fe9e8c04cb5b5b25d5a92a6914f, State: Initialized, Role: FOLLOWER
I20260806 22:46:33.807250 2547 consensus_queue.cc:260] T 00000000000000000000000000000000 P d5736fe9e8c04cb5b5b25d5a92a6914f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5736fe9e8c04cb5b5b25d5a92a6914f" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 33077 } }
I20260806 22:46:33.807355 2547 raft_consensus.cc:388] T 00000000000000000000000000000000 P d5736fe9e8c04cb5b5b25d5a92a6914f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:46:33.807433 2547 raft_consensus.cc:482] T 00000000000000000000000000000000 P d5736fe9e8c04cb5b5b25d5a92a6914f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:46:33.807518 2547 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d5736fe9e8c04cb5b5b25d5a92a6914f [term 0 FOLLOWER]: Advancing to term 1
Aug 06 22:46:33 dist-test-slave-h3ft krb5kdc[1762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056393, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:46:33.808449 2547 raft_consensus.cc:504] T 00000000000000000000000000000000 P d5736fe9e8c04cb5b5b25d5a92a6914f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5736fe9e8c04cb5b5b25d5a92a6914f" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 33077 } }
I20260806 22:46:33.808570 2547 leader_election.cc:302] T 00000000000000000000000000000000 P d5736fe9e8c04cb5b5b25d5a92a6914f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d5736fe9e8c04cb5b5b25d5a92a6914f; no voters:
I20260806 22:46:33.808792 2547 leader_election.cc:288] T 00000000000000000000000000000000 P d5736fe9e8c04cb5b5b25d5a92a6914f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:46:33.808876 2556 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d5736fe9e8c04cb5b5b25d5a92a6914f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:46:33.809247 2547 sys_catalog.cc:564] T 00000000000000000000000000000000 P d5736fe9e8c04cb5b5b25d5a92a6914f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:46:33.809491 2556 raft_consensus.cc:686] T 00000000000000000000000000000000 P d5736fe9e8c04cb5b5b25d5a92a6914f [term 1 LEADER]: Becoming Leader. State: Replica: d5736fe9e8c04cb5b5b25d5a92a6914f, State: Running, Role: LEADER
I20260806 22:46:33.809689 2556 consensus_queue.cc:237] T 00000000000000000000000000000000 P d5736fe9e8c04cb5b5b25d5a92a6914f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5736fe9e8c04cb5b5b25d5a92a6914f" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 33077 } }
I20260806 22:46:33.810494 2556 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5736fe9e8c04cb5b5b25d5a92a6914f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d5736fe9e8c04cb5b5b25d5a92a6914f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5736fe9e8c04cb5b5b25d5a92a6914f" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 33077 } } }
I20260806 22:46:33.810655 2556 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5736fe9e8c04cb5b5b25d5a92a6914f [sys.catalog]: This master's current role is: LEADER
I20260806 22:46:33.810892 2556 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5736fe9e8c04cb5b5b25d5a92a6914f [sys.catalog]: SysCatalogTable state changed. Reason: New leader d5736fe9e8c04cb5b5b25d5a92a6914f. Latest consensus state: current_term: 1 leader_uuid: "d5736fe9e8c04cb5b5b25d5a92a6914f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5736fe9e8c04cb5b5b25d5a92a6914f" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 33077 } } }
I20260806 22:46:33.811040 2556 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5736fe9e8c04cb5b5b25d5a92a6914f [sys.catalog]: This master's current role is: LEADER
I20260806 22:46:33.811260 2574 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:46:33.812121 2574 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:46:33.811273 2547 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 22:46:33.813571 2552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:33.800550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49100 (local address 127.18.45.254:33077)
0806 22:46:33.800857 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:33.800869 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:33.800916 (+ 47us) server_negotiation.cc:407] Connection header received
0806 22:46:33.801708 (+ 792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:33.801719 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:33.801801 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:33.802007 (+ 206us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:33.802953 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:33.804172 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:33.805044 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:33.805566 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:33.805708 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:33.808739 (+ 3031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:33.808744 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:33.808744 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:33.808757 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:33.811186 (+ 2429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:33.811627 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:33.811635 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:33.811636 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:33.811761 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:33.812024 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:33.812025 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:33.812026 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:33.812388 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:33.812580 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:33.813169 (+ 589us) server_negotiation.cc:299] Negotiation successful
0806 22:46:33.813304 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":159,"threads_started":1}
I20260806 22:46:33.821141 2574 catalog_manager.cc:1347] Generated new cluster ID: bea4ee9654df4e019ad609d5592300c6
I20260806 22:46:33.821364 2574 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:46:33.830754 2574 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:46:33.831807 2574 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:46:33.841387 2574 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d5736fe9e8c04cb5b5b25d5a92a6914f: Generated new TSK 0
I20260806 22:46:33.841816 2574 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.45.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab.
Entry for principal kudu/127.18.45.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab.
I20260806 22:46:34.680017 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.45.212:55641/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.193:0
--local_ip_for_outbound_sockets=127.18.45.193
--webserver_interface=127.18.45.193
--webserver_port=0
--tserver_master_addrs=127.18.45.254:33077
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab
--principal=kudu/127.18.45.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:32783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:46:34.826859 2620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:46:34.827265 2620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:46:34.827314 2620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:46:34.829273 2620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:46:34.829361 2620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:46:34.829413 2620 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.193
I20260806 22:46:34.832037 2620 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:32783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.18.45.212:55641/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab
--principal=kudu/127.18.45.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.45.193
--webserver_port=0
--tserver_master_addrs=127.18.45.254:33077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2620
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.193
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:46:34.832366 2620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:46:34.832772 2620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:46:34.840334 2629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:34.840497 2631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:34.840349 2628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:34.841121 2620 server_base.cc:1034] running on GCE node
I20260806 22:46:34.841367 2620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:46:34.841729 2620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:46:34.844992 2620 hybrid_clock.cc:648] HybridClock initialized: now 1786056394844438 us; error 559 us; skew 500 ppm
Aug 06 22:46:34 dist-test-slave-h3ft krb5kdc[1762](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056394, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:46:34.849517 2620 init.cc:377] Logged in from keytab as kudu/127.18.45.193@KRBTEST.COM (short username kudu)
I20260806 22:46:34.850242 2620 webserver.cc:460] Webserver started at http://127.18.45.193:40399/ using document root <none> and password file <none>
I20260806 22:46:34.850518 2620 fs_manager.cc:359] Metadata directory not provided
I20260806 22:46:34.850597 2620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:46:34.850765 2620 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 22:46:34 dist-test-slave-h3ft krb5kdc[1762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056394, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.193@KRBTEST.COM for HTTP/127.18.45.212@KRBTEST.COM
I20260806 22:46:34.868566 2620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/ts-0/data/instance:
uuid: "5dbcb9a7596d40b0bf5c0664f638a414"
format_stamp: "Formatted at 2026-08-06 22:46:34 on dist-test-slave-h3ft"
server_key: "7a63ed48f701bb6da7495c61d216f006"
server_key_iv: "91e58372190f777fbc1a5c8c25569c69"
server_key_version: "kuduclusterkey@0"
I20260806 22:46:34.869285 2620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/ts-0/wal/instance:
uuid: "5dbcb9a7596d40b0bf5c0664f638a414"
format_stamp: "Formatted at 2026-08-06 22:46:34 on dist-test-slave-h3ft"
server_key: "7a63ed48f701bb6da7495c61d216f006"
server_key_iv: "91e58372190f777fbc1a5c8c25569c69"
server_key_version: "kuduclusterkey@0"
I20260806 22:46:34.871731 2620 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260806 22:46:34.889676 2644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:34.890038 2620 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:46:34.890223 2620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/ts-0/wal
uuid: "5dbcb9a7596d40b0bf5c0664f638a414"
format_stamp: "Formatted at 2026-08-06 22:46:34 on dist-test-slave-h3ft"
server_key: "7a63ed48f701bb6da7495c61d216f006"
server_key_iv: "91e58372190f777fbc1a5c8c25569c69"
server_key_version: "kuduclusterkey@0"
I20260806 22:46:34.890413 2620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:46:34.915957 2620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:46:34.919023 2620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:46:34.919284 2620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:46:34.919703 2620 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:46:34.920262 2620 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:46:34.920364 2620 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:34.920501 2620 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:46:34.920607 2620 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:34.930230 2620 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.193:40825
I20260806 22:46:34.930959 2620 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
I20260806 22:46:34.936224 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 2620
I20260806 22:46:34.936537 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/ts-0/wal/instance
I20260806 22:46:34.936808 18615 external_mini_cluster.cc:1428] Setting key 5049c762dd2b91478d63764bf83cda2c
I20260806 22:46:34.932389 2814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.193:40825 every 8 connection(s)
Aug 06 22:46:34 dist-test-slave-h3ft krb5kdc[1762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056394, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.193@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.45.194@KRBTEST.COM; defaulting to no policy
I20260806 22:46:34.951741 2819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:34.933427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:40005 (local address 127.18.45.254:33077)
0806 22:46:34.936403 (+ 2976us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:34.936406 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:34.936417 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:46:34.936571 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:34.936573 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:34.936643 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:34.936731 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:34.941639 (+ 4908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:34.942563 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:34.943970 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:34.944191 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:34.944295 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:34.947321 (+ 3026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:34.947323 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:34.947323 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:34.947335 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:34.949612 (+ 2277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:34.950417 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:34.950419 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:34.950419 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:34.950507 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:34.950845 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:34.950846 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:34.950847 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:34.951066 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:34.951149 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:34.951496 (+ 347us) server_negotiation.cc:299] Negotiation successful
0806 22:46:34.951609 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2937,"thread_start_us":82,"threads_started":1}
I20260806 22:46:34.951990 2821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:34.933392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:33077 (local address 127.18.45.193:40005)
0806 22:46:34.935203 (+ 1811us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:34.935279 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:34.936144 (+ 865us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:34.940649 (+ 4505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:34.940660 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:34.941033 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:46:34.941445 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:34.941459 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:34.942713 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:34.942714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:34.943829 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:34.943833 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:34.944336 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:34.944337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:34.944515 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:34.945223 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:46:34.945234 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:46:34.947181 (+ 1947us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:46:34.949741 (+ 2560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:34.949751 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:34.949755 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:34.950297 (+ 542us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:34.950593 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:34.950594 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:34.950594 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:34.950756 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:34.951138 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:46:34.951147 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:46:34.951332 (+ 185us) client_negotiation.cc:769] Sending connection context
0806 22:46:34.951458 (+ 126us) client_negotiation.cc:240] Negotiation successful
0806 22:46:34.951662 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1741,"thread_start_us":113,"threads_started":1}
I20260806 22:46:34.952909 2816 heartbeater.cc:347] Connected to a master server at 127.18.45.254:33077
I20260806 22:46:34.953092 2816 heartbeater.cc:464] Registering TS with master...
I20260806 22:46:34.953377 2816 heartbeater.cc:511] Master 127.18.45.254:33077 requested a full tablet report, sending...
I20260806 22:46:34.954214 2496 ts_manager.cc:194] Registered new tserver with Master: 5dbcb9a7596d40b0bf5c0664f638a414 (127.18.45.193:40825)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.194@KRBTEST.COM" created.
I20260806 22:46:34.955276 2496 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.45.193@KRBTEST.COM'} at 127.18.45.193:40005
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab.
Entry for principal kudu/127.18.45.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab.
I20260806 22:46:34.982493 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.45.212:55641/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.194:0
--local_ip_for_outbound_sockets=127.18.45.194
--webserver_interface=127.18.45.194
--webserver_port=0
--tserver_master_addrs=127.18.45.254:33077
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab
--principal=kudu/127.18.45.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:32783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:46:35.108503 2824 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:46:35.108819 2824 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:46:35.108866 2824 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:46:35.110754 2824 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:46:35.110831 2824 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:46:35.110888 2824 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.194
I20260806 22:46:35.113037 2824 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:32783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.18.45.212:55641/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab
--principal=kudu/127.18.45.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.45.194
--webserver_port=0
--tserver_master_addrs=127.18.45.254:33077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2824
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.194
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:46:35.113386 2824 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:46:35.113701 2824 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:46:35.120566 2836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:35.120572 2833 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:35.120708 2840 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:35.121868 2824 server_base.cc:1034] running on GCE node
I20260806 22:46:35.122154 2824 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:46:35.122521 2824 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:46:35.123732 2824 hybrid_clock.cc:648] HybridClock initialized: now 1786056395123713 us; error 61 us; skew 500 ppm
Aug 06 22:46:35 dist-test-slave-h3ft krb5kdc[1762](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056395, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:46:35.127908 2824 init.cc:377] Logged in from keytab as kudu/127.18.45.194@KRBTEST.COM (short username kudu)
I20260806 22:46:35.128648 2824 webserver.cc:460] Webserver started at http://127.18.45.194:40389/ using document root <none> and password file <none>
I20260806 22:46:35.129019 2824 fs_manager.cc:359] Metadata directory not provided
I20260806 22:46:35.129113 2824 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:46:35.129283 2824 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 22:46:35 dist-test-slave-h3ft krb5kdc[1762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056395, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.194@KRBTEST.COM for HTTP/127.18.45.212@KRBTEST.COM
Aug 06 22:46:35 dist-test-slave-h3ft krb5kdc[1762](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786056395, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:46:35.145097 2824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/ts-1/data/instance:
uuid: "4787b19e69244409a97c01f55f23cd77"
format_stamp: "Formatted at 2026-08-06 22:46:35 on dist-test-slave-h3ft"
server_key: "abb3bc8efe51254ebcc7c34fbd27dcf7"
server_key_iv: "7dfa18992b797b7407b13c9d64a5129b"
server_key_version: "kuduclusterkey@0"
I20260806 22:46:35.145905 2824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/ts-1/wal/instance:
uuid: "4787b19e69244409a97c01f55f23cd77"
format_stamp: "Formatted at 2026-08-06 22:46:35 on dist-test-slave-h3ft"
server_key: "abb3bc8efe51254ebcc7c34fbd27dcf7"
server_key_iv: "7dfa18992b797b7407b13c9d64a5129b"
server_key_version: "kuduclusterkey@0"
I20260806 22:46:35.148322 2824 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260806 22:46:35.184295 2849 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:35.187791 2824 fs_manager.cc:725] Time spent opening block manager: real 0.015s user 0.001s sys 0.001s
I20260806 22:46:35.188041 2824 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/ts-1/wal
uuid: "4787b19e69244409a97c01f55f23cd77"
format_stamp: "Formatted at 2026-08-06 22:46:35 on dist-test-slave-h3ft"
server_key: "abb3bc8efe51254ebcc7c34fbd27dcf7"
server_key_iv: "7dfa18992b797b7407b13c9d64a5129b"
server_key_version: "kuduclusterkey@0"
I20260806 22:46:35.188210 2824 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:46:35.208362 2824 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:46:35.211519 2824 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:46:35.211745 2824 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:46:35.212117 2824 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:46:35.212599 2824 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:46:35.212662 2824 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:35.212740 2824 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:46:35.212770 2824 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:35.222523 2824 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.194:34023
I20260806 22:46:35.223138 2824 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
I20260806 22:46:35.223347 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 2824
I20260806 22:46:35.223507 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/ts-1/wal/instance
I20260806 22:46:35.223779 18615 external_mini_cluster.cc:1428] Setting key 819996a4d47b0f6496ede965970df6dd
I20260806 22:46:35.232676 3004 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.194:34023 every 8 connection(s)
WARNING: no policy specified for kudu/127.18.45.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.195@KRBTEST.COM" created.
Aug 06 22:46:35 dist-test-slave-h3ft krb5kdc[1762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056395, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.194@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:46:35.252333 3009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:35.236042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:33077 (local address 127.18.45.194:42125)
0806 22:46:35.236787 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:35.236845 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:35.237580 (+ 735us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:35.237996 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:35.238000 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:35.238239 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:46:35.238587 (+ 348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:35.238594 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.239674 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.239675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:35.242234 (+ 2559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:35.242237 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.242856 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.242857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:35.243063 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:35.245553 (+ 2490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:46:35.245568 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:46:35.247714 (+ 2146us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:46:35.250156 (+ 2442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:35.250166 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:35.250170 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:35.250560 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:35.250837 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:35.250838 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:35.250838 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:35.251070 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:35.251452 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:46:35.251460 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:46:35.251672 (+ 212us) client_negotiation.cc:769] Sending connection context
0806 22:46:35.251798 (+ 126us) client_negotiation.cc:240] Negotiation successful
0806 22:46:35.252009 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":674,"thread_start_us":153,"threads_started":1}
I20260806 22:46:35.253059 2819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:35.236521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:42125 (local address 127.18.45.254:33077)
0806 22:46:35.236610 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:35.236613 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:35.236976 (+ 363us) server_negotiation.cc:407] Connection header received
0806 22:46:35.237721 (+ 745us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:35.237723 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:35.237748 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:35.237784 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:35.238713 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.239554 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.242360 (+ 2806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.242693 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.242742 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:35.247874 (+ 5132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:35.247876 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:35.247876 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:35.247888 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:35.250025 (+ 2137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:35.250670 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:35.250672 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:35.250672 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:35.250756 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:35.251163 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:35.251165 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:35.251165 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:35.251353 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:35.251423 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:35.252787 (+ 1364us) server_negotiation.cc:299] Negotiation successful
0806 22:46:35.252910 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 22:46:35.253943 3005 heartbeater.cc:347] Connected to a master server at 127.18.45.254:33077
I20260806 22:46:35.254082 3005 heartbeater.cc:464] Registering TS with master...
I20260806 22:46:35.254345 3005 heartbeater.cc:511] Master 127.18.45.254:33077 requested a full tablet report, sending...
I20260806 22:46:35.254805 2496 ts_manager.cc:194] Registered new tserver with Master: 4787b19e69244409a97c01f55f23cd77 (127.18.45.194:34023)
I20260806 22:46:35.255407 2496 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.45.194@KRBTEST.COM'} at 127.18.45.194:42125
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab.
Entry for principal kudu/127.18.45.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab.
I20260806 22:46:35.278486 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.45.212:55641/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.195:0
--local_ip_for_outbound_sockets=127.18.45.195
--webserver_interface=127.18.45.195
--webserver_port=0
--tserver_master_addrs=127.18.45.254:33077
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab
--principal=kudu/127.18.45.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:32783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:46:35.398806 3016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:46:35.399168 3016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:46:35.399222 3016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:46:35.400789 3016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:46:35.400887 3016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:46:35.400939 3016 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.195
I20260806 22:46:35.402647 3016 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:32783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.18.45.212:55641/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab
--principal=kudu/127.18.45.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.45.195
--webserver_port=0
--tserver_master_addrs=127.18.45.254:33077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3016
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.195
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:46:35.403016 3016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:46:35.403307 3016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:46:35.409379 3024 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:35.409386 3022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:46:35.410012 3026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:46:35.410507 3016 server_base.cc:1034] running on GCE node
I20260806 22:46:35.410781 3016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:46:35.411185 3016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:46:35.412367 3016 hybrid_clock.cc:648] HybridClock initialized: now 1786056395412351 us; error 52 us; skew 500 ppm
Aug 06 22:46:35 dist-test-slave-h3ft krb5kdc[1762](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056395, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:46:35.415763 3016 init.cc:377] Logged in from keytab as kudu/127.18.45.195@KRBTEST.COM (short username kudu)
I20260806 22:46:35.416409 3016 webserver.cc:460] Webserver started at http://127.18.45.195:45059/ using document root <none> and password file <none>
I20260806 22:46:35.416776 3016 fs_manager.cc:359] Metadata directory not provided
I20260806 22:46:35.416865 3016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:46:35.417026 3016 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 22:46:35 dist-test-slave-h3ft krb5kdc[1762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056395, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.195@KRBTEST.COM for HTTP/127.18.45.212@KRBTEST.COM
I20260806 22:46:35.444650 3016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/ts-2/data/instance:
uuid: "4a14223f27404284871ddf24bf73a7ec"
format_stamp: "Formatted at 2026-08-06 22:46:35 on dist-test-slave-h3ft"
server_key: "fccb0b5689bb058c9eefbd2dba7a8ec3"
server_key_iv: "05debf6ac534a6ac3a3608dcdcfbd15d"
server_key_version: "kuduclusterkey@0"
I20260806 22:46:35.445304 3016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/ts-2/wal/instance:
uuid: "4a14223f27404284871ddf24bf73a7ec"
format_stamp: "Formatted at 2026-08-06 22:46:35 on dist-test-slave-h3ft"
server_key: "fccb0b5689bb058c9eefbd2dba7a8ec3"
server_key_iv: "05debf6ac534a6ac3a3608dcdcfbd15d"
server_key_version: "kuduclusterkey@0"
I20260806 22:46:35.449144 3016 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 22:46:35.480945 3042 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:35.481283 3016 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.000s
I20260806 22:46:35.481472 3016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/ts-2/wal
uuid: "4a14223f27404284871ddf24bf73a7ec"
format_stamp: "Formatted at 2026-08-06 22:46:35 on dist-test-slave-h3ft"
server_key: "fccb0b5689bb058c9eefbd2dba7a8ec3"
server_key_iv: "05debf6ac534a6ac3a3608dcdcfbd15d"
server_key_version: "kuduclusterkey@0"
I20260806 22:46:35.481634 3016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:46:35.505813 3016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:46:35.508690 3016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:46:35.508958 3016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:46:35.509367 3016 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:46:35.509856 3016 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:46:35.509958 3016 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:35.510067 3016 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:46:35.510114 3016 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:46:35.524603 3016 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.195:33887
I20260806 22:46:35.524948 3207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.195:33887 every 8 connection(s)
I20260806 22:46:35.525378 3016 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
I20260806 22:46:35.531931 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 3016
I20260806 22:46:35.532086 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/ts-2/wal/instance
I20260806 22:46:35.532305 18615 external_mini_cluster.cc:1428] Setting key d6e1217ca3912fa6b4c597079050a4e9
Aug 06 22:46:35 dist-test-slave-h3ft krb5kdc[1762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056395, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.195@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:46:35.543397 2819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:35.526403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.195:41571 (local address 127.18.45.254:33077)
0806 22:46:35.526471 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:35.526474 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:35.531447 (+ 4973us) server_negotiation.cc:407] Connection header received
0806 22:46:35.532825 (+ 1378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:35.532827 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:35.532843 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:35.532897 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:35.533670 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.534233 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.535460 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.535676 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.535764 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:35.538987 (+ 3223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:35.538990 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:35.538990 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:35.539028 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:35.541165 (+ 2137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:35.541948 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:35.541950 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:35.541951 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:35.542037 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:35.542412 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:35.542413 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:35.542414 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:35.542612 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:35.542706 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:35.543153 (+ 447us) server_negotiation.cc:299] Negotiation successful
0806 22:46:35.543268 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 22:46:35.543632 3212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:35.526362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:33077 (local address 127.18.45.195:41571)
0806 22:46:35.531191 (+ 4829us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:35.531280 (+ 89us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:35.532455 (+ 1175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:35.533013 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:35.533019 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:35.533278 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:46:35.533551 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:35.533555 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.534341 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.534343 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:35.535362 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:35.535365 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.535787 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.535788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:35.535954 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:35.536720 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:46:35.536735 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:46:35.538839 (+ 2104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:46:35.541455 (+ 2616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:35.541466 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:35.541469 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:35.541845 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:35.542127 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:46:35.542128 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:46:35.542129 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:46:35.542305 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:46:35.542705 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:46:35.542713 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:46:35.542927 (+ 214us) client_negotiation.cc:769] Sending connection context
0806 22:46:35.543084 (+ 157us) client_negotiation.cc:240] Negotiation successful
0806 22:46:35.543300 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4747,"thread_start_us":108,"threads_started":1}
I20260806 22:46:35.544310 3210 heartbeater.cc:347] Connected to a master server at 127.18.45.254:33077
I20260806 22:46:35.544442 3210 heartbeater.cc:464] Registering TS with master...
I20260806 22:46:35.544683 3210 heartbeater.cc:511] Master 127.18.45.254:33077 requested a full tablet report, sending...
I20260806 22:46:35.545202 2496 ts_manager.cc:194] Registered new tserver with Master: 4a14223f27404284871ddf24bf73a7ec (127.18.45.195:33887)
I20260806 22:46:35.545814 2496 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.45.195@KRBTEST.COM'} at 127.18.45.195:41571
I20260806 22:46:35.555693 18615 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:46:35.570434 2819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:35.557959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49116 (local address 127.18.45.254:33077)
0806 22:46:35.558017 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:35.558020 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:35.558210 (+ 190us) server_negotiation.cc:407] Connection header received
0806 22:46:35.558255 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:35.558256 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:35.558271 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:35.558352 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:46:35.559235 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.560105 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.561741 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.562108 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.562199 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:35.564528 (+ 2329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:46:35.564530 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:46:35.564530 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:46:35.564541 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:46:35.566724 (+ 2183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:35.567240 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:35.567241 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:35.567242 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:35.567326 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:46:35.569592 (+ 2266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:46:35.569594 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:46:35.569594 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:46:35.569785 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:46:35.569909 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:35.570175 (+ 266us) server_negotiation.cc:299] Negotiation successful
0806 22:46:35.570282 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 22:46:35.573024 2496 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49116:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:46:35.575702 2496 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:46:35.585773 3233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:35.579332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.195:33887 (local address 127.0.0.1:50198)
0806 22:46:35.580363 (+ 1031us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:35.580420 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:35.580723 (+ 303us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:35.581123 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:35.581126 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:35.581130 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:46:35.581229 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:35.581232 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.582812 (+ 1580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.582813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:35.584286 (+ 1473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:35.584287 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.585407 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.585408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:35.585502 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:35.585523 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 22:46:35.585578 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 22:46:35.585648 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":985,"mutex_wait_us":2,"thread_start_us":148,"threads_started":1}
I20260806 22:46:35.586407 3232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:35.579652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.194:34023 (local address 127.0.0.1:52344)
0806 22:46:35.581055 (+ 1403us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:35.581065 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:35.581108 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:35.581660 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:35.581661 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:35.581664 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:46:35.581715 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:35.581717 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.583193 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.583194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:35.584283 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:35.584287 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.585994 (+ 1707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.585995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:35.586057 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:35.586062 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:46:35.586110 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 22:46:35.586156 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1382,"thread_start_us":86,"threads_started":1}
I20260806 22:46:35.586603 3239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:35.580104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50198 (local address 127.18.45.195:33887)
0806 22:46:35.580653 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:35.580659 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:35.580673 (+ 14us) server_negotiation.cc:407] Connection header received
0806 22:46:35.580871 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:35.580875 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:35.580900 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:35.580992 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:46:35.581318 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.582727 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.584444 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.585291 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.586295 (+ 1004us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:35.586413 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:35.586447 (+ 34us) server_negotiation.cc:299] Negotiation successful
0806 22:46:35.586506 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":492,"thread_start_us":115,"threads_started":1}
I20260806 22:46:35.587302 3238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:35.580096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52344 (local address 127.18.45.194:34023)
0806 22:46:35.581250 (+ 1154us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:35.581254 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:35.581265 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:46:35.581312 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:35.581314 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:35.581331 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:35.581546 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:46:35.581796 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.583100 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.584439 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.585761 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.585829 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:35.585951 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:35.587152 (+ 1201us) server_negotiation.cc:299] Negotiation successful
0806 22:46:35.587208 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1106,"thread_start_us":118,"threads_started":1}
I20260806 22:46:35.587656 3109 tablet_service.cc:1467] Processing CreateTablet for tablet c6f44b0493fd4ba58ab844b1870f1602 (DEFAULT_TABLE table=test-table [id=165d33c0d87b4953bda574f8cb011b3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:46:35.588197 3109 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6f44b0493fd4ba58ab844b1870f1602. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:35.592716 3231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:35.579332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.193:40825 (local address 127.0.0.1:48748)
0806 22:46:35.580907 (+ 1575us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:35.580917 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:35.580969 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:35.588273 (+ 7304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:35.588275 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:35.588277 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:46:35.588334 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:35.588335 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.589739 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.589741 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:35.590903 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:35.590908 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.592410 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.592411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:35.592494 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:35.592501 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 22:46:35.592537 (+ 36us) client_negotiation.cc:240] Negotiation successful
0806 22:46:35.592620 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1553,"mutex_wait_us":2,"thread_start_us":147,"threads_started":1}
I20260806 22:46:35.593235 3234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:35.579730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48748 (local address 127.18.45.193:40825)
0806 22:46:35.587997 (+ 8267us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:35.588003 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:35.588017 (+ 14us) server_negotiation.cc:407] Connection header received
0806 22:46:35.588063 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:35.588065 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:35.588087 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:35.588422 (+ 335us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:46:35.588519 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.589621 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.591452 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.592296 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.592923 (+ 627us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:35.593012 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:35.593046 (+ 34us) server_negotiation.cc:299] Negotiation successful
0806 22:46:35.593096 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8209,"thread_start_us":83,"threads_started":1}
I20260806 22:46:35.593559 2895 tablet_service.cc:1467] Processing CreateTablet for tablet c6f44b0493fd4ba58ab844b1870f1602 (DEFAULT_TABLE table=test-table [id=165d33c0d87b4953bda574f8cb011b3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:46:35.594031 2895 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6f44b0493fd4ba58ab844b1870f1602. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:35.593918 2716 tablet_service.cc:1467] Processing CreateTablet for tablet c6f44b0493fd4ba58ab844b1870f1602 (DEFAULT_TABLE table=test-table [id=165d33c0d87b4953bda574f8cb011b3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:46:35.594290 2716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6f44b0493fd4ba58ab844b1870f1602. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:46:35.598825 3244 tablet_bootstrap.cc:492] T c6f44b0493fd4ba58ab844b1870f1602 P 5dbcb9a7596d40b0bf5c0664f638a414: Bootstrap starting.
I20260806 22:46:35.599022 3245 tablet_bootstrap.cc:492] T c6f44b0493fd4ba58ab844b1870f1602 P 4787b19e69244409a97c01f55f23cd77: Bootstrap starting.
I20260806 22:46:35.599948 3244 tablet_bootstrap.cc:654] T c6f44b0493fd4ba58ab844b1870f1602 P 5dbcb9a7596d40b0bf5c0664f638a414: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:35.600332 3245 tablet_bootstrap.cc:654] T c6f44b0493fd4ba58ab844b1870f1602 P 4787b19e69244409a97c01f55f23cd77: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:35.600461 3244 log.cc:824] T c6f44b0493fd4ba58ab844b1870f1602 P 5dbcb9a7596d40b0bf5c0664f638a414: Log is configured to *not* fsync() on all Append() calls
I20260806 22:46:35.600832 3245 log.cc:824] T c6f44b0493fd4ba58ab844b1870f1602 P 4787b19e69244409a97c01f55f23cd77: Log is configured to *not* fsync() on all Append() calls
I20260806 22:46:35.601146 3242 tablet_bootstrap.cc:492] T c6f44b0493fd4ba58ab844b1870f1602 P 4a14223f27404284871ddf24bf73a7ec: Bootstrap starting.
I20260806 22:46:35.601509 3244 tablet_bootstrap.cc:492] T c6f44b0493fd4ba58ab844b1870f1602 P 5dbcb9a7596d40b0bf5c0664f638a414: No bootstrap required, opened a new log
I20260806 22:46:35.601598 3244 ts_tablet_manager.cc:1397] T c6f44b0493fd4ba58ab844b1870f1602 P 5dbcb9a7596d40b0bf5c0664f638a414: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:46:35.602660 3242 tablet_bootstrap.cc:654] T c6f44b0493fd4ba58ab844b1870f1602 P 4a14223f27404284871ddf24bf73a7ec: Neither blocks nor log segments found. Creating new log.
I20260806 22:46:35.603121 3242 log.cc:824] T c6f44b0493fd4ba58ab844b1870f1602 P 4a14223f27404284871ddf24bf73a7ec: Log is configured to *not* fsync() on all Append() calls
I20260806 22:46:35.604708 3242 tablet_bootstrap.cc:492] T c6f44b0493fd4ba58ab844b1870f1602 P 4a14223f27404284871ddf24bf73a7ec: No bootstrap required, opened a new log
I20260806 22:46:35.604789 3242 ts_tablet_manager.cc:1397] T c6f44b0493fd4ba58ab844b1870f1602 P 4a14223f27404284871ddf24bf73a7ec: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260806 22:46:35.606400 3244 raft_consensus.cc:348] T c6f44b0493fd4ba58ab844b1870f1602 P 5dbcb9a7596d40b0bf5c0664f638a414 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dbcb9a7596d40b0bf5c0664f638a414" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 40825 } } peers { permanent_uuid: "4a14223f27404284871ddf24bf73a7ec" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 33887 } } peers { permanent_uuid: "4787b19e69244409a97c01f55f23cd77" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 34023 } }
I20260806 22:46:35.606781 3244 raft_consensus.cc:374] T c6f44b0493fd4ba58ab844b1870f1602 P 5dbcb9a7596d40b0bf5c0664f638a414 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:35.606901 3244 raft_consensus.cc:729] T c6f44b0493fd4ba58ab844b1870f1602 P 5dbcb9a7596d40b0bf5c0664f638a414 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5dbcb9a7596d40b0bf5c0664f638a414, State: Initialized, Role: FOLLOWER
I20260806 22:46:35.607173 3244 consensus_queue.cc:260] T c6f44b0493fd4ba58ab844b1870f1602 P 5dbcb9a7596d40b0bf5c0664f638a414 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dbcb9a7596d40b0bf5c0664f638a414" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 40825 } } peers { permanent_uuid: "4a14223f27404284871ddf24bf73a7ec" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 33887 } } peers { permanent_uuid: "4787b19e69244409a97c01f55f23cd77" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 34023 } }
I20260806 22:46:35.607193 3242 raft_consensus.cc:348] T c6f44b0493fd4ba58ab844b1870f1602 P 4a14223f27404284871ddf24bf73a7ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dbcb9a7596d40b0bf5c0664f638a414" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 40825 } } peers { permanent_uuid: "4a14223f27404284871ddf24bf73a7ec" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 33887 } } peers { permanent_uuid: "4787b19e69244409a97c01f55f23cd77" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 34023 } }
I20260806 22:46:35.607385 3242 raft_consensus.cc:374] T c6f44b0493fd4ba58ab844b1870f1602 P 4a14223f27404284871ddf24bf73a7ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:35.607471 3242 raft_consensus.cc:729] T c6f44b0493fd4ba58ab844b1870f1602 P 4a14223f27404284871ddf24bf73a7ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a14223f27404284871ddf24bf73a7ec, State: Initialized, Role: FOLLOWER
I20260806 22:46:35.607690 3244 ts_tablet_manager.cc:1428] T c6f44b0493fd4ba58ab844b1870f1602 P 5dbcb9a7596d40b0bf5c0664f638a414: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260806 22:46:35.607688 3242 consensus_queue.cc:260] T c6f44b0493fd4ba58ab844b1870f1602 P 4a14223f27404284871ddf24bf73a7ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dbcb9a7596d40b0bf5c0664f638a414" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 40825 } } peers { permanent_uuid: "4a14223f27404284871ddf24bf73a7ec" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 33887 } } peers { permanent_uuid: "4787b19e69244409a97c01f55f23cd77" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 34023 } }
I20260806 22:46:35.607897 2816 heartbeater.cc:503] Master 127.18.45.254:33077 was elected leader, sending a full tablet report...
I20260806 22:46:35.608057 3242 ts_tablet_manager.cc:1428] T c6f44b0493fd4ba58ab844b1870f1602 P 4a14223f27404284871ddf24bf73a7ec: Time spent starting tablet: real 0.003s user 0.003s sys 0.001s
I20260806 22:46:35.608167 3210 heartbeater.cc:503] Master 127.18.45.254:33077 was elected leader, sending a full tablet report...
I20260806 22:46:35.609305 3245 tablet_bootstrap.cc:492] T c6f44b0493fd4ba58ab844b1870f1602 P 4787b19e69244409a97c01f55f23cd77: No bootstrap required, opened a new log
I20260806 22:46:35.609447 3245 ts_tablet_manager.cc:1397] T c6f44b0493fd4ba58ab844b1870f1602 P 4787b19e69244409a97c01f55f23cd77: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.000s
I20260806 22:46:35.611747 3245 raft_consensus.cc:348] T c6f44b0493fd4ba58ab844b1870f1602 P 4787b19e69244409a97c01f55f23cd77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dbcb9a7596d40b0bf5c0664f638a414" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 40825 } } peers { permanent_uuid: "4a14223f27404284871ddf24bf73a7ec" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 33887 } } peers { permanent_uuid: "4787b19e69244409a97c01f55f23cd77" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 34023 } }
I20260806 22:46:35.611948 3245 raft_consensus.cc:374] T c6f44b0493fd4ba58ab844b1870f1602 P 4787b19e69244409a97c01f55f23cd77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:46:35.612017 3245 raft_consensus.cc:729] T c6f44b0493fd4ba58ab844b1870f1602 P 4787b19e69244409a97c01f55f23cd77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4787b19e69244409a97c01f55f23cd77, State: Initialized, Role: FOLLOWER
I20260806 22:46:35.612169 3245 consensus_queue.cc:260] T c6f44b0493fd4ba58ab844b1870f1602 P 4787b19e69244409a97c01f55f23cd77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dbcb9a7596d40b0bf5c0664f638a414" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 40825 } } peers { permanent_uuid: "4a14223f27404284871ddf24bf73a7ec" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 33887 } } peers { permanent_uuid: "4787b19e69244409a97c01f55f23cd77" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 34023 } }
I20260806 22:46:35.612644 3245 ts_tablet_manager.cc:1428] T c6f44b0493fd4ba58ab844b1870f1602 P 4787b19e69244409a97c01f55f23cd77: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:46:35.612807 3005 heartbeater.cc:503] Master 127.18.45.254:33077 was elected leader, sending a full tablet report...
I20260806 22:46:35.675648 3259 raft_consensus.cc:482] T c6f44b0493fd4ba58ab844b1870f1602 P 4787b19e69244409a97c01f55f23cd77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:46:35.675884 3259 raft_consensus.cc:504] T c6f44b0493fd4ba58ab844b1870f1602 P 4787b19e69244409a97c01f55f23cd77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dbcb9a7596d40b0bf5c0664f638a414" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 40825 } } peers { permanent_uuid: "4a14223f27404284871ddf24bf73a7ec" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 33887 } } peers { permanent_uuid: "4787b19e69244409a97c01f55f23cd77" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 34023 } }
I20260806 22:46:35.677083 3259 leader_election.cc:288] T c6f44b0493fd4ba58ab844b1870f1602 P 4787b19e69244409a97c01f55f23cd77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5dbcb9a7596d40b0bf5c0664f638a414 (127.18.45.193:40825), 4a14223f27404284871ddf24bf73a7ec (127.18.45.195:33887)
I20260806 22:46:35.681747 3009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:35.676666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.193:40825 (local address 127.18.45.194:58889)
0806 22:46:35.676814 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:35.676830 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:35.676889 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:35.677859 (+ 970us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:35.677860 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:35.677862 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:46:35.677917 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:35.677918 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.678857 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.678858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:35.680401 (+ 1543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:35.680406 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.681381 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.681382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:35.681479 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:35.681484 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:46:35.681536 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 22:46:35.681617 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":60}
I20260806 22:46:35.681998 3239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:35.676997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:49185 (local address 127.18.45.195:33887)
0806 22:46:35.677054 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:35.677057 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:35.677420 (+ 363us) server_negotiation.cc:407] Connection header received
0806 22:46:35.677552 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:35.677554 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:35.677566 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:35.677597 (+ 31us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:46:35.677874 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.678716 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.680922 (+ 2206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.681553 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.681603 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:35.681613 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:35.681839 (+ 226us) server_negotiation.cc:299] Negotiation successful
0806 22:46:35.681900 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 22:46:35.682126 3234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:35.677043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:58889 (local address 127.18.45.193:40825)
0806 22:46:35.677203 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:35.677205 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:35.677217 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:46:35.677257 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:35.677258 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:35.677270 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:35.677327 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:46:35.678023 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.678755 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.680572 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.681251 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.681332 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:35.681345 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:35.681970 (+ 625us) server_negotiation.cc:299] Negotiation successful
0806 22:46:35.682041 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127}
I20260806 22:46:35.682719 2747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6f44b0493fd4ba58ab844b1870f1602" candidate_uuid: "4787b19e69244409a97c01f55f23cd77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dbcb9a7596d40b0bf5c0664f638a414" is_pre_election: true
I20260806 22:46:35.682979 2747 raft_consensus.cc:2454] T c6f44b0493fd4ba58ab844b1870f1602 P 5dbcb9a7596d40b0bf5c0664f638a414 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4787b19e69244409a97c01f55f23cd77 in term 0.
I20260806 22:46:35.683502 2851 leader_election.cc:302] T c6f44b0493fd4ba58ab844b1870f1602 P 4787b19e69244409a97c01f55f23cd77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4787b19e69244409a97c01f55f23cd77, 5dbcb9a7596d40b0bf5c0664f638a414; no voters:
I20260806 22:46:35.683669 3259 raft_consensus.cc:2781] T c6f44b0493fd4ba58ab844b1870f1602 P 4787b19e69244409a97c01f55f23cd77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:46:35.683732 3259 raft_consensus.cc:482] T c6f44b0493fd4ba58ab844b1870f1602 P 4787b19e69244409a97c01f55f23cd77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:46:35.683753 3259 raft_consensus.cc:3037] T c6f44b0493fd4ba58ab844b1870f1602 P 4787b19e69244409a97c01f55f23cd77 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:35.684206 3260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:35.676666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.195:33887 (local address 127.18.45.194:49185)
0806 22:46:35.677351 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:35.677363 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:46:35.677450 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:46:35.677676 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:46:35.677676 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:46:35.677679 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:46:35.677787 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:35.677790 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.678857 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.678858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:35.680810 (+ 1952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:46:35.680813 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.681673 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.681673 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:46:35.681751 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:35.681755 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:46:35.684068 (+ 2313us) client_negotiation.cc:240] Negotiation successful
0806 22:46:35.684127 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":662,"mutex_wait_us":674,"thread_start_us":152,"threads_started":1}
W20260806 22:46:35.684384 2818 tablet.cc:2369] T c6f44b0493fd4ba58ab844b1870f1602 P 5dbcb9a7596d40b0bf5c0664f638a414: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:46:35.684739 3137 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6f44b0493fd4ba58ab844b1870f1602" candidate_uuid: "4787b19e69244409a97c01f55f23cd77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a14223f27404284871ddf24bf73a7ec" is_pre_election: true
I20260806 22:46:35.684989 3137 raft_consensus.cc:2454] T c6f44b0493fd4ba58ab844b1870f1602 P 4a14223f27404284871ddf24bf73a7ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4787b19e69244409a97c01f55f23cd77 in term 0.
I20260806 22:46:35.685091 3259 raft_consensus.cc:504] T c6f44b0493fd4ba58ab844b1870f1602 P 4787b19e69244409a97c01f55f23cd77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dbcb9a7596d40b0bf5c0664f638a414" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 40825 } } peers { permanent_uuid: "4a14223f27404284871ddf24bf73a7ec" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 33887 } } peers { permanent_uuid: "4787b19e69244409a97c01f55f23cd77" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 34023 } }
I20260806 22:46:35.685261 3259 leader_election.cc:288] T c6f44b0493fd4ba58ab844b1870f1602 P 4787b19e69244409a97c01f55f23cd77 [CANDIDATE]: Term 1 election: Requested vote from peers 5dbcb9a7596d40b0bf5c0664f638a414 (127.18.45.193:40825), 4a14223f27404284871ddf24bf73a7ec (127.18.45.195:33887)
I20260806 22:46:35.685715 3137 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6f44b0493fd4ba58ab844b1870f1602" candidate_uuid: "4787b19e69244409a97c01f55f23cd77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a14223f27404284871ddf24bf73a7ec"
I20260806 22:46:35.685751 2747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6f44b0493fd4ba58ab844b1870f1602" candidate_uuid: "4787b19e69244409a97c01f55f23cd77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dbcb9a7596d40b0bf5c0664f638a414"
I20260806 22:46:35.685837 2747 raft_consensus.cc:3037] T c6f44b0493fd4ba58ab844b1870f1602 P 5dbcb9a7596d40b0bf5c0664f638a414 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:35.685837 3137 raft_consensus.cc:3037] T c6f44b0493fd4ba58ab844b1870f1602 P 4a14223f27404284871ddf24bf73a7ec [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:46:35.686970 2747 raft_consensus.cc:2454] T c6f44b0493fd4ba58ab844b1870f1602 P 5dbcb9a7596d40b0bf5c0664f638a414 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4787b19e69244409a97c01f55f23cd77 in term 1.
I20260806 22:46:35.686971 3137 raft_consensus.cc:2454] T c6f44b0493fd4ba58ab844b1870f1602 P 4a14223f27404284871ddf24bf73a7ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4787b19e69244409a97c01f55f23cd77 in term 1.
I20260806 22:46:35.687229 2851 leader_election.cc:302] T c6f44b0493fd4ba58ab844b1870f1602 P 4787b19e69244409a97c01f55f23cd77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4787b19e69244409a97c01f55f23cd77, 5dbcb9a7596d40b0bf5c0664f638a414; no voters:
I20260806 22:46:35.687355 3259 raft_consensus.cc:2781] T c6f44b0493fd4ba58ab844b1870f1602 P 4787b19e69244409a97c01f55f23cd77 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:46:35.687554 3259 raft_consensus.cc:686] T c6f44b0493fd4ba58ab844b1870f1602 P 4787b19e69244409a97c01f55f23cd77 [term 1 LEADER]: Becoming Leader. State: Replica: 4787b19e69244409a97c01f55f23cd77, State: Running, Role: LEADER
I20260806 22:46:35.687700 3259 consensus_queue.cc:237] T c6f44b0493fd4ba58ab844b1870f1602 P 4787b19e69244409a97c01f55f23cd77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dbcb9a7596d40b0bf5c0664f638a414" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 40825 } } peers { permanent_uuid: "4a14223f27404284871ddf24bf73a7ec" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 33887 } } peers { permanent_uuid: "4787b19e69244409a97c01f55f23cd77" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 34023 } }
I20260806 22:46:35.688921 2493 catalog_manager.cc:5495] T c6f44b0493fd4ba58ab844b1870f1602 P 4787b19e69244409a97c01f55f23cd77 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4787b19e69244409a97c01f55f23cd77 (127.18.45.194). New cstate: current_term: 1 leader_uuid: "4787b19e69244409a97c01f55f23cd77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dbcb9a7596d40b0bf5c0664f638a414" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 40825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a14223f27404284871ddf24bf73a7ec" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 33887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4787b19e69244409a97c01f55f23cd77" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 34023 } health_report { overall_health: HEALTHY } } }
I20260806 22:46:35.708747 3238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:35.704637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52350 (local address 127.18.45.194:34023)
0806 22:46:35.704688 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 22:46:35.704691 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:46:35.704699 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:46:35.704760 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:46:35.704761 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:46:35.704790 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:46:35.704866 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:46:35.705108 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.706074 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.707381 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:46:35.707710 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:46:35.707783 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:46:35.708017 (+ 234us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:46:35.708204 (+ 187us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:46:35.708295 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 22:46:35.708609 (+ 314us) server_negotiation.cc:299] Negotiation successful
0806 22:46:35.708640 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 22:46:35.711879 2747 raft_consensus.cc:1261] T c6f44b0493fd4ba58ab844b1870f1602 P 5dbcb9a7596d40b0bf5c0664f638a414 [term 1 FOLLOWER]: Refusing update from remote peer 4787b19e69244409a97c01f55f23cd77: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:46:35.711879 3137 raft_consensus.cc:1261] T c6f44b0493fd4ba58ab844b1870f1602 P 4a14223f27404284871ddf24bf73a7ec [term 1 FOLLOWER]: Refusing update from remote peer 4787b19e69244409a97c01f55f23cd77: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:46:35.712412 3259 consensus_queue.cc:1035] T c6f44b0493fd4ba58ab844b1870f1602 P 4787b19e69244409a97c01f55f23cd77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a14223f27404284871ddf24bf73a7ec" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 33887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:46:35.712674 3259 consensus_queue.cc:1035] T c6f44b0493fd4ba58ab844b1870f1602 P 4787b19e69244409a97c01f55f23cd77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dbcb9a7596d40b0bf5c0664f638a414" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 40825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:46:35.718053 3266 mvcc.cc:205] Tried to move back new op lower bound from 7315686996831543296 to 7315686996738228224. Current Snapshot: MvccSnapshot[applied={T|T < 7315686996831543296}]
I20260806 22:46:35.719440 3267 mvcc.cc:205] Tried to move back new op lower bound from 7315686996831543296 to 7315686996738228224. Current Snapshot: MvccSnapshot[applied={T|T < 7315686996831543296}]
Aug 06 22:46:35 dist-test-slave-h3ft krb5kdc[1762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056395, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.254@KRBTEST.COM for HTTP/127.18.45.212@KRBTEST.COM
I20260806 22:46:35.731070 2493 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49116:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:46:35.731230 2493 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49116:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:46:35.731504 3270 mvcc.cc:205] Tried to move back new op lower bound from 7315686996831543296 to 7315686996738228224. Current Snapshot: MvccSnapshot[applied={T|T < 7315686996831543296}]
I20260806 22:46:35.732343 2493 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d5736fe9e8c04cb5b5b25d5a92a6914f: Sending DeleteTablet for 3 replicas of tablet c6f44b0493fd4ba58ab844b1870f1602
I20260806 22:46:35.732858 3109 tablet_service.cc:1514] Processing DeleteTablet for tablet c6f44b0493fd4ba58ab844b1870f1602 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:46:35 UTC) from {username='kudu', principal='kudu/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:50198
I20260806 22:46:35.732892 2895 tablet_service.cc:1514] Processing DeleteTablet for tablet c6f44b0493fd4ba58ab844b1870f1602 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:46:35 UTC) from {username='kudu', principal='kudu/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:52344
I20260806 22:46:35.733723 2716 tablet_service.cc:1514] Processing DeleteTablet for tablet c6f44b0493fd4ba58ab844b1870f1602 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:46:35 UTC) from {username='kudu', principal='kudu/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:48748
I20260806 22:46:35.734503 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 2620
I20260806 22:46:35.754033 3277 tablet_replica.cc:330] T c6f44b0493fd4ba58ab844b1870f1602 P 4787b19e69244409a97c01f55f23cd77: stopping tablet replica
I20260806 22:46:35.754365 3277 raft_consensus.cc:2229] T c6f44b0493fd4ba58ab844b1870f1602 P 4787b19e69244409a97c01f55f23cd77 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:46:35.754761 3277 raft_consensus.cc:2258] T c6f44b0493fd4ba58ab844b1870f1602 P 4787b19e69244409a97c01f55f23cd77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:46:35.755645 3277 ts_tablet_manager.cc:1905] T c6f44b0493fd4ba58ab844b1870f1602 P 4787b19e69244409a97c01f55f23cd77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:46:35.757987 3277 ts_tablet_manager.cc:1918] T c6f44b0493fd4ba58ab844b1870f1602 P 4787b19e69244409a97c01f55f23cd77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:46:35.758164 3277 log.cc:1196] T c6f44b0493fd4ba58ab844b1870f1602 P 4787b19e69244409a97c01f55f23cd77: Deleting WAL directory at /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/ts-1/wal/wals/c6f44b0493fd4ba58ab844b1870f1602
I20260806 22:46:35.758596 3277 ts_tablet_manager.cc:1939] T c6f44b0493fd4ba58ab844b1870f1602 P 4787b19e69244409a97c01f55f23cd77: Deleting consensus metadata
I20260806 22:46:35.759375 2468 catalog_manager.cc:4841] TS 4787b19e69244409a97c01f55f23cd77 (127.18.45.194:34023): tablet c6f44b0493fd4ba58ab844b1870f1602 (table test-table [id=165d33c0d87b4953bda574f8cb011b3a]) successfully deleted
W20260806 22:46:35.763777 2466 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.45.193:40825 (error 108)
I20260806 22:46:35.764302 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 2824
I20260806 22:46:35.771710 3279 tablet_replica.cc:330] T c6f44b0493fd4ba58ab844b1870f1602 P 4a14223f27404284871ddf24bf73a7ec: stopping tablet replica
I20260806 22:46:35.771955 3279 raft_consensus.cc:2229] T c6f44b0493fd4ba58ab844b1870f1602 P 4a14223f27404284871ddf24bf73a7ec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:46:35.772157 3279 raft_consensus.cc:2258] T c6f44b0493fd4ba58ab844b1870f1602 P 4a14223f27404284871ddf24bf73a7ec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:46:35.774153 3279 ts_tablet_manager.cc:1905] T c6f44b0493fd4ba58ab844b1870f1602 P 4a14223f27404284871ddf24bf73a7ec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:46:35.776860 3279 ts_tablet_manager.cc:1918] T c6f44b0493fd4ba58ab844b1870f1602 P 4a14223f27404284871ddf24bf73a7ec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:46:35.777026 3279 log.cc:1196] T c6f44b0493fd4ba58ab844b1870f1602 P 4a14223f27404284871ddf24bf73a7ec: Deleting WAL directory at /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786056275803658-18615-0/minicluster-data/ts-2/wal/wals/c6f44b0493fd4ba58ab844b1870f1602
I20260806 22:46:35.777783 3279 ts_tablet_manager.cc:1939] T c6f44b0493fd4ba58ab844b1870f1602 P 4a14223f27404284871ddf24bf73a7ec: Deleting consensus metadata
I20260806 22:46:35.778477 2468 catalog_manager.cc:4841] TS 4a14223f27404284871ddf24bf73a7ec (127.18.45.195:33887): tablet c6f44b0493fd4ba58ab844b1870f1602 (table test-table [id=165d33c0d87b4953bda574f8cb011b3a]) successfully deleted
I20260806 22:46:35.782343 3231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:46:35.782139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.193:40825 (local address 127.0.0.1:48752)
0806 22:46:35.782209 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0806 22:46:35.782284 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.193:40825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260806 22:46:35.782701 2466 catalog_manager.cc:4584] TS 5dbcb9a7596d40b0bf5c0664f638a414 (127.18.45.193:40825): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c6f44b0493fd4ba58ab844b1870f1602: Network error: Client connection negotiation failed: client connection to 127.18.45.193:40825: connect: Connection refused (error 111)
I20260806 22:46:35.785272 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 3016
I20260806 22:46:35.796023 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 2440
I20260806 22:46:35.821027 18615 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 22:46:36.176999 18615 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 22:46:36.177150 18615 mini_ranger.cc:66] Stopping Ranger...
I20260806 22:46:36.542214 18615 mini_ranger.cc:68] Stopped Ranger
2026-08-06 22:46:36.542 UTC [1807] LOG: received smart shutdown request
2026-08-06 22:46:36.547 UTC [1807] LOG: background worker "logical replication launcher" (PID 1820) exited with exit code 1
2026-08-06 22:46:36.547 UTC [1811] LOG: shutting down
2026-08-06 22:46:36.730 UTC [1807] LOG: database system is shut down
2026-08-06T22:46:36Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (55418 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:46:36 dist-test-slave-h3ft krb5kdc[3295](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:46:36 dist-test-slave-h3ft krb5kdc[3295](info): set up 2 sockets
Aug 06 22:46:36 dist-test-slave-h3ft krb5kdc[3295](info): commencing operation
krb5kdc: starting...
W20260806 22:46:38.126024 18615 mini_kdc.cc:121] Time spent starting KDC: real 1.241s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Aug 06 22:46:38 dist-test-slave-h3ft krb5kdc[3295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056398, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:46:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:46:38Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.18.45.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.18.45.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.18.45.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/krb5kdc/rangeradmin_127.18.45.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.18.45.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/krb5kdc/rangeradmin_127.18.45.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.18.45.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.18.45.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.18.45.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/krb5kdc/rangerlookup_127.18.45.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.18.45.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/krb5kdc/rangerlookup_127.18.45.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.18.45.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.18.45.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.18.45.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/krb5kdc/HTTP_127.18.45.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.18.45.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/krb5kdc/HTTP_127.18.45.212@KRBTEST.COM.keytab.
I20260806 22:46:38.318490 18615 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskXSWSdD/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/postgres -l logfile start
2026-08-06 22:46:41.126 UTC [3335] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 22:46:41.126 UTC [3335] LOG: listening on IPv4 address "127.18.45.212", port 33859
2026-08-06 22:46:41.130 UTC [3335] LOG: listening on Unix socket "/tmp/.s.PGSQL.33859"
2026-08-06 22:46:41.155 UTC [3339] LOG: database system was shut down at 2026-08-06 22:46:39 UTC
2026-08-06 22:46:41.160 UTC [3335] LOG: database system is ready to accept connections
I20260806 22:46:42.373575 18615 mini_postgres.cc:96] Postgres bound to 33859
2026-08-06 22:46:42.389 UTC [3352] FATAL: database "slave" does not exist
127.18.45.212:33859 - accepting connections
I20260806 22:46:42.390381 18615 mini_ranger.cc:161] Starting Ranger...
I20260806 22:46:42.414330 18615 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 22:46:43.001978 18615 mini_ranger.cc:87] Created ranger Postgres database
I20260806 22:46:43.002100 18615 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ranger-admin
2026-08-06 22:46:43,316 [I] DB FLAVOR :POSTGRES
2026-08-06 22:46:43,317 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 22:46:43,317 [I] Checking connection..
2026-08-06 22:46:43,317 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXSWSdD/build/release/bin/postgresql.jar:/tmp/dist-test-taskXSWSdD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.45.212:33859/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 22:46:43,629 [I] Checking connection passed.
2026-08-06 22:46:43,629 [I] --------- Verifying version history table ---------
2026-08-06 22:46:43,630 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXSWSdD/build/release/bin/postgresql.jar:/tmp/dist-test-taskXSWSdD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.45.212:33859/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 22:46:43,921 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 22:46:43,922 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXSWSdD/build/release/bin/postgresql.jar:/tmp/dist-test-taskXSWSdD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.45.212:33859/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 22:46:44,216 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 22:46:44,216 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 22:46:44,216 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXSWSdD/build/release/bin/postgresql.jar:/tmp/dist-test-taskXSWSdD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.45.212:33859/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskXSWSdD/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 22:46:44.519 UTC [3449] WARNING: there is no transaction in progress
2026-08-06 22:46:44,529 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 22:46:44,529 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXSWSdD/build/release/bin/postgresql.jar:/tmp/dist-test-taskXSWSdD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.45.212:33859/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 22:46:44,842 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 22:46:44,843 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 22:46:44,843 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXSWSdD/build/release/bin/postgresql.jar:/tmp/dist-test-taskXSWSdD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.45.212:33859/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 22:46:45,140 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXSWSdD/build/release/bin/postgresql.jar:/tmp/dist-test-taskXSWSdD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.45.212:33859/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 22:46:45,447 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXSWSdD/build/release/bin/postgresql.jar:/tmp/dist-test-taskXSWSdD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.45.212:33859/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-h3ft.c.gcp-upstream.internal','N') ;"
2026-08-06 22:46:45,734 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXSWSdD/build/release/bin/postgresql.jar:/tmp/dist-test-taskXSWSdD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.45.212:33859/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 22:46:46,053 [I] Table x_portal_user does not exist in database ranger
2026-08-06 22:46:46,054 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXSWSdD/build/release/bin/postgresql.jar:/tmp/dist-test-taskXSWSdD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.45.212:33859/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 22:46:46,329 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 22:46:46,329 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXSWSdD/build/release/bin/postgresql.jar:/tmp/dist-test-taskXSWSdD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.45.212:33859/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 22:46:46,651 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXSWSdD/build/release/bin/postgresql.jar:/tmp/dist-test-taskXSWSdD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.45.212:33859/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 22:46:46,952 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 22:46:46,953 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXSWSdD/build/release/bin/postgresql.jar:/tmp/dist-test-taskXSWSdD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.45.212:33859/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskXSWSdD/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 22:46:48.148 UTC [3611] WARNING: there is no transaction in progress
2026-08-06 22:46:48.168 UTC [3611] WARNING: there is no transaction in progress
2026-08-06 22:46:48.188 UTC [3611] WARNING: there is no transaction in progress
2026-08-06 22:46:48.207 UTC [3611] WARNING: there is no transaction in progress
2026-08-06 22:46:48.226 UTC [3611] WARNING: there is no transaction in progress
2026-08-06 22:46:48.250 UTC [3611] WARNING: there is no transaction in progress
2026-08-06 22:46:48.319 UTC [3611] WARNING: there is no transaction in progress
2026-08-06 22:46:48.328 UTC [3611] WARNING: there is no transaction in progress
2026-08-06 22:46:48.348 UTC [3611] WARNING: there is no transaction in progress
2026-08-06 22:46:48.360 UTC [3611] WARNING: there is no transaction in progress
2026-08-06 22:46:48.372 UTC [3611] WARNING: there is no transaction in progress
2026-08-06 22:46:48.390 UTC [3611] WARNING: there is no transaction in progress
2026-08-06 22:46:48.403 UTC [3611] WARNING: there is no transaction in progress
2026-08-06 22:46:48.412 UTC [3611] WARNING: there is no transaction in progress
2026-08-06 22:46:48.421 UTC [3611] WARNING: there is no transaction in progress
2026-08-06 22:46:48.430 UTC [3611] WARNING: there is no transaction in progress
2026-08-06 22:46:49,253 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 22:46:49,253 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXSWSdD/build/release/bin/postgresql.jar:/tmp/dist-test-taskXSWSdD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.45.212:33859/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 22:46:49,562 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 22:46:49,563 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXSWSdD/build/release/bin/postgresql.jar:/tmp/dist-test-taskXSWSdD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.45.212:33859/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 22:46:49,856 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 22:46:49,857 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXSWSdD/build/release/bin/postgresql.jar:/tmp/dist-test-taskXSWSdD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.45.212:33859/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 22:46:50,171 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 22:46:50,172 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXSWSdD/build/release/bin/postgresql.jar:/tmp/dist-test-taskXSWSdD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.45.212:33859/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 22:46:50,464 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXSWSdD/build/release/bin/postgresql.jar:/tmp/dist-test-taskXSWSdD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.45.212:33859/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 22:46:50,773 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXSWSdD/build/release/bin/postgresql.jar:/tmp/dist-test-taskXSWSdD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.45.212:33859/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-h3ft.c.gcp-upstream.internal';"
2026-08-06 22:46:51,101 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 22:46:51,101 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXSWSdD/build/release/bin/postgresql.jar:/tmp/dist-test-taskXSWSdD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.45.212:33859/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 22:46:51,396 [I] DB_PATCHES have already been applied
I20260806 22:46:51.407632 18615 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ranger-admin:/tmp/dist-test-taskXSWSdD/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskXSWSdD/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskXSWSdD/build/release/bin/postgresql.jar:/tmp/dist-test-taskXSWSdD/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 22:46:51.407747 18615 mini_ranger.cc:193] Using host: 127.18.45.212
I20260806 22:46:51.414248 18615 mini_ranger.cc:227] Ranger admin URL: http://127.18.45.212:50345
Aug 06, 2026 10:46:51 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 10:46:51 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 10:46:51 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 10:46:51 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 10:46:51 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 10:46:52 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 10:46:52 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 22:46:52.768723 18615 mini_ranger.cc:160] Time spent starting Ranger: real 10.378s user 0.000s sys 0.015s
Aug 06, 2026 10:47:16 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 10:47:16 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 06, 2026 10:47:16 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 10:47:16 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 10:47:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 10:47:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 10:47:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 10:47:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 10:47:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 10:47:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 10:47:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 10:47:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 10:47:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 10:47:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 10:47:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 10:47:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 10:47:16 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 10:47:17 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 22:47:18.456642 18615 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.18.45.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.18.45.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.18.45.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/krb5kdc/rangerkms_127.18.45.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.18.45.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/krb5kdc/rangerkms_127.18.45.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.18.45.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.18.45.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.18.45.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/krb5kdc/HTTP_127.18.45.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.18.45.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/krb5kdc/HTTP_127.18.45.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 22:47:18 dist-test-slave-h3ft krb5kdc[3295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056438, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 22:47:18.565945 18615 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 22:47:18.581312 18615 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 22:47:19.568048 18615 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 22:47:19.568182 18615 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ranger-kms
I20260806 22:47:19.568205 18615 mini_ranger_kms.cc:227] Using postgres at 127.18.45.212:33859
2026-08-06 22:47:19,793 [I] DB FLAVOR :POSTGRES
2026-08-06 22:47:19,794 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 22:47:19,794 [I] Checking connection
2026-08-06 22:47:19,794 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXSWSdD/build/release/bin/postgresql.jar:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.45.212:33859/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 22:47:20,095 [I] connection success
2026-08-06 22:47:20,096 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 22:47:20,096 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 22:47:20,096 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXSWSdD/build/release/bin/postgresql.jar:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.45.212:33859/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 22:47:20,396 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 22:47:20,397 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 22:47:20,397 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 22:47:20,397 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXSWSdD/build/release/bin/postgresql.jar:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.45.212:33859/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 22:47:20,709 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 22:47:21.294183 18615 mini_ranger_kms.cc:310] Created kms service
I20260806 22:47:21.511917 18615 mini_ranger_kms.cc:326] Created kudu user
I20260806 22:47:21.567989 18615 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 22:47:21 dist-test-slave-h3ft krb5kdc[3295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056438, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.18.45.212@KRBTEST.COM
I20260806 22:47:22.078307 18615 mini_ranger_kms.cc:384] Added ranger policy
I20260806 22:47:22.078612 18615 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ranger-kms:/tmp/dist-test-taskXSWSdD/build/release/bin/postgresql.jar:/tmp/dist-test-taskXSWSdD/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskXSWSdD/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 22:47:22.078648 18615 mini_ranger_kms.cc:240] Using host: 127.18.45.212
I20260806 22:47:22.087714 18615 mini_ranger_kms.cc:276] Ranger KMS PID: 3872
I20260806 22:47:22.087850 18615 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.18.45.212:55743
Aug 06, 2026 10:47:23 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 10:47:23 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 10:47:23 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 10:47:23 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 10:47:23 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 10:47:23 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 06 22:47:24 dist-test-slave-h3ft krb5kdc[3295](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786056444, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.45.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:24.829566 18615 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.264s user 0.000s sys 0.209s
I20260806 22:47:24.829700 18615 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 22:47:24.829766 18615 mini_ranger_kms.cc:401] 127.18.45.212:55743/kms/v1/keys
Aug 06 22:47:25 dist-test-slave-h3ft krb5kdc[3295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056444, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.45.212@KRBTEST.COM for HTTP/127.18.45.212@KRBTEST.COM
Aug 06 22:47:25 dist-test-slave-h3ft krb5kdc[3295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056444, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.45.212@KRBTEST.COM for HTTP/127.18.45.212@KRBTEST.COM
Aug 06 22:47:28 dist-test-slave-h3ft krb5kdc[3295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056448, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.18.45.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab.
Entry for principal kudu/127.18.45.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab.
I20260806 22:47:28.354743 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:43077
--webserver_interface=127.18.45.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:35547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.45.254:43077
--ranger_config_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.45.212:55743/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:47:28.453146 3942 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:28.453433 3942 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:28.453460 3942 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:28.454721 3942 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:47:28.454777 3942 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:28.454794 3942 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:28.454808 3942 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:47:28.454829 3942 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:47:28.456662 3942 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:35547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/master-0/wal
--ranger_kms_url=127.18.45.212:55743/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.18.45.254:43077
--ranger_config_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:43077
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.45.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3942
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:47:28.456996 3942 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:28.457295 3942 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:28.464613 3954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:28.464613 3942 server_base.cc:1034] running on GCE node
W20260806 22:47:28.464653 3949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:28.464852 3960 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:28.465230 3942 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:28.465605 3942 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:28.466804 3942 hybrid_clock.cc:648] HybridClock initialized: now 1786056448466777 us; error 46 us; skew 500 ppm
Aug 06 22:47:28 dist-test-slave-h3ft krb5kdc[3295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056448, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:28.477929 3942 init.cc:377] Logged in from keytab as kudu/127.18.45.254@KRBTEST.COM (short username kudu)
I20260806 22:47:28.478718 3942 webserver.cc:460] Webserver started at http://127.18.45.254:41731/ using document root <none> and password file <none>
I20260806 22:47:28.479058 3942 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:28.479139 3942 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:28.479308 3942 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 22:47:28 dist-test-slave-h3ft krb5kdc[3295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056448, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.254@KRBTEST.COM for HTTP/127.18.45.212@KRBTEST.COM
I20260806 22:47:28.547153 3942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/master-0/data/instance:
uuid: "0c72db8fbc444b13bf1f88a9a4762d66"
format_stamp: "Formatted at 2026-08-06 22:47:28 on dist-test-slave-h3ft"
server_key: "0ad36734c764ac640e5bc4f049761032"
server_key_iv: "c7306fc02e56118ebcaf3525d8fe9397"
server_key_version: "kuduclusterkey@0"
I20260806 22:47:28.547714 3942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/master-0/wal/instance:
uuid: "0c72db8fbc444b13bf1f88a9a4762d66"
format_stamp: "Formatted at 2026-08-06 22:47:28 on dist-test-slave-h3ft"
server_key: "0ad36734c764ac640e5bc4f049761032"
server_key_iv: "c7306fc02e56118ebcaf3525d8fe9397"
server_key_version: "kuduclusterkey@0"
I20260806 22:47:28.549808 3942 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 22:47:28.568470 3970 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:28.568856 3942 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 22:47:28.569010 3942 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/master-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/master-0/wal
uuid: "0c72db8fbc444b13bf1f88a9a4762d66"
format_stamp: "Formatted at 2026-08-06 22:47:28 on dist-test-slave-h3ft"
server_key: "0ad36734c764ac640e5bc4f049761032"
server_key_iv: "c7306fc02e56118ebcaf3525d8fe9397"
server_key_version: "kuduclusterkey@0"
I20260806 22:47:28.569146 3942 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:28.606698 3942 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:28.609462 3942 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:28.609650 3942 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:28.615607 3942 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.254:43077
I20260806 22:47:28.615665 4045 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.254:43077 every 8 connection(s)
I20260806 22:47:28.616158 3942 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
I20260806 22:47:28.617133 4046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:47:28.620191 4046 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c72db8fbc444b13bf1f88a9a4762d66: Bootstrap starting.
I20260806 22:47:28.621025 4046 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c72db8fbc444b13bf1f88a9a4762d66: Neither blocks nor log segments found. Creating new log.
I20260806 22:47:28.621409 4046 log.cc:824] T 00000000000000000000000000000000 P 0c72db8fbc444b13bf1f88a9a4762d66: Log is configured to *not* fsync() on all Append() calls
I20260806 22:47:28.622308 4046 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c72db8fbc444b13bf1f88a9a4762d66: No bootstrap required, opened a new log
I20260806 22:47:28.624181 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 3942
I20260806 22:47:28.624317 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/master-0/wal/instance
I20260806 22:47:28.624531 18615 external_mini_cluster.cc:1428] Setting key 20f94d1eed4e864e2471eeda635c3a18
I20260806 22:47:28.625801 4046 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c72db8fbc444b13bf1f88a9a4762d66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c72db8fbc444b13bf1f88a9a4762d66" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 43077 } }
I20260806 22:47:28.626139 4046 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0c72db8fbc444b13bf1f88a9a4762d66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:47:28.626214 4046 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c72db8fbc444b13bf1f88a9a4762d66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c72db8fbc444b13bf1f88a9a4762d66, State: Initialized, Role: FOLLOWER
I20260806 22:47:28.626401 4046 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c72db8fbc444b13bf1f88a9a4762d66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c72db8fbc444b13bf1f88a9a4762d66" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 43077 } }
I20260806 22:47:28.626533 4046 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0c72db8fbc444b13bf1f88a9a4762d66 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:47:28.626598 4046 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c72db8fbc444b13bf1f88a9a4762d66 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:47:28.626662 4046 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c72db8fbc444b13bf1f88a9a4762d66 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:47:28.627683 4046 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c72db8fbc444b13bf1f88a9a4762d66 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c72db8fbc444b13bf1f88a9a4762d66" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 43077 } }
I20260806 22:47:28.627825 4046 leader_election.cc:302] T 00000000000000000000000000000000 P 0c72db8fbc444b13bf1f88a9a4762d66 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c72db8fbc444b13bf1f88a9a4762d66; no voters:
I20260806 22:47:28.628074 4046 leader_election.cc:288] T 00000000000000000000000000000000 P 0c72db8fbc444b13bf1f88a9a4762d66 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:47:28.628129 4051 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0c72db8fbc444b13bf1f88a9a4762d66 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:47:28.628439 4051 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0c72db8fbc444b13bf1f88a9a4762d66 [term 1 LEADER]: Becoming Leader. State: Replica: 0c72db8fbc444b13bf1f88a9a4762d66, State: Running, Role: LEADER
I20260806 22:47:28.628532 4046 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c72db8fbc444b13bf1f88a9a4762d66 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:47:28.628597 4051 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c72db8fbc444b13bf1f88a9a4762d66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c72db8fbc444b13bf1f88a9a4762d66" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 43077 } }
I20260806 22:47:28.629027 4051 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c72db8fbc444b13bf1f88a9a4762d66 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0c72db8fbc444b13bf1f88a9a4762d66" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c72db8fbc444b13bf1f88a9a4762d66" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 43077 } } }
I20260806 22:47:28.629163 4051 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c72db8fbc444b13bf1f88a9a4762d66 [sys.catalog]: This master's current role is: LEADER
I20260806 22:47:28.629335 4051 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c72db8fbc444b13bf1f88a9a4762d66 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c72db8fbc444b13bf1f88a9a4762d66. Latest consensus state: current_term: 1 leader_uuid: "0c72db8fbc444b13bf1f88a9a4762d66" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c72db8fbc444b13bf1f88a9a4762d66" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 43077 } } }
I20260806 22:47:28.629443 4051 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c72db8fbc444b13bf1f88a9a4762d66 [sys.catalog]: This master's current role is: LEADER
I20260806 22:47:28.629611 4060 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:47:28.630234 4060 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:47:28.630616 4046 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 22:47:28.633277 4060 catalog_manager.cc:1347] Generated new cluster ID: c577159029074be2ac8a0e2ffe316a44
I20260806 22:47:28.633332 4060 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:47:28 dist-test-slave-h3ft krb5kdc[3295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056448, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:47:28.643745 4050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:28.625271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60348 (local address 127.18.45.254:43077)
0806 22:47:28.625518 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:28.625534 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:28.625576 (+ 42us) server_negotiation.cc:407] Connection header received
0806 22:47:28.625752 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:28.625768 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:28.625845 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:28.626037 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:28.626916 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:28.628717 (+ 1801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:28.630199 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:28.630985 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:28.631564 (+ 579us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:28.634557 (+ 2993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:28.634563 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:28.634564 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:28.634581 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:28.637093 (+ 2512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:28.637601 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:28.637605 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:28.637606 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:28.637768 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:28.638026 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:28.638027 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:28.638028 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:28.638423 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:28.638595 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:28.639266 (+ 671us) server_negotiation.cc:299] Negotiation successful
0806 22:47:28.642531 (+ 3265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":133,"threads_started":1}
I20260806 22:47:28.653920 4060 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:47:28.654759 4060 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:47:28.661715 4060 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0c72db8fbc444b13bf1f88a9a4762d66: Generated new TSK 0
I20260806 22:47:28.662030 4060 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.45.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab.
Entry for principal kudu/127.18.45.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab.
I20260806 22:47:29.534200 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.45.212:55743/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.193:0
--local_ip_for_outbound_sockets=127.18.45.193
--webserver_interface=127.18.45.193
--webserver_port=0
--tserver_master_addrs=127.18.45.254:43077
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab
--principal=kudu/127.18.45.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:35547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:47:29.644583 4109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:29.644879 4109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:29.644927 4109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:29.646427 4109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:29.646509 4109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:29.646562 4109 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.193
I20260806 22:47:29.648643 4109 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:35547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.18.45.212:55743/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab
--principal=kudu/127.18.45.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.45.193
--webserver_port=0
--tserver_master_addrs=127.18.45.254:43077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4109
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.193
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:47:29.648945 4109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:29.649273 4109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:29.655643 4118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:29.655686 4121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:29.655983 4117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:29.656255 4109 server_base.cc:1034] running on GCE node
I20260806 22:47:29.656515 4109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:29.656809 4109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:29.657994 4109 hybrid_clock.cc:648] HybridClock initialized: now 1786056449657963 us; error 49 us; skew 500 ppm
Aug 06 22:47:29 dist-test-slave-h3ft krb5kdc[3295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056449, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:29.661335 4109 init.cc:377] Logged in from keytab as kudu/127.18.45.193@KRBTEST.COM (short username kudu)
I20260806 22:47:29.662159 4109 webserver.cc:460] Webserver started at http://127.18.45.193:40581/ using document root <none> and password file <none>
I20260806 22:47:29.662513 4109 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:29.662570 4109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:29.662755 4109 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 22:47:29 dist-test-slave-h3ft krb5kdc[3295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056449, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.193@KRBTEST.COM for HTTP/127.18.45.212@KRBTEST.COM
I20260806 22:47:29.682617 4109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-0/data/instance:
uuid: "eb8153447c6e45a2ab4d3360bae8c43a"
format_stamp: "Formatted at 2026-08-06 22:47:29 on dist-test-slave-h3ft"
server_key: "c40b8fb13f41c19e3088d5a86d8a1196"
server_key_iv: "dcae938ce40c963ba1b6ec56f0d930c0"
server_key_version: "kuduclusterkey@0"
I20260806 22:47:29.683257 4109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-0/wal/instance:
uuid: "eb8153447c6e45a2ab4d3360bae8c43a"
format_stamp: "Formatted at 2026-08-06 22:47:29 on dist-test-slave-h3ft"
server_key: "c40b8fb13f41c19e3088d5a86d8a1196"
server_key_iv: "dcae938ce40c963ba1b6ec56f0d930c0"
server_key_version: "kuduclusterkey@0"
I20260806 22:47:29.686069 4109 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.002s
I20260806 22:47:29.704422 4139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:29.704704 4109 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:47:29.704825 4109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-0/wal
uuid: "eb8153447c6e45a2ab4d3360bae8c43a"
format_stamp: "Formatted at 2026-08-06 22:47:29 on dist-test-slave-h3ft"
server_key: "c40b8fb13f41c19e3088d5a86d8a1196"
server_key_iv: "dcae938ce40c963ba1b6ec56f0d930c0"
server_key_version: "kuduclusterkey@0"
I20260806 22:47:29.704970 4109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:29.728456 4109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:29.731630 4109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:29.731848 4109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:29.732179 4109 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:47:29.732717 4109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:47:29.732779 4109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:29.732820 4109 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:47:29.732887 4109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:29.743163 4109 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.193:40131
I20260806 22:47:29.743234 4295 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.193:40131 every 8 connection(s)
I20260806 22:47:29.743851 4109 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
I20260806 22:47:29.753746 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 4109
I20260806 22:47:29.753897 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-0/wal/instance
I20260806 22:47:29.754199 18615 external_mini_cluster.cc:1428] Setting key ee21a59b156bebb41aa2ff8247a03bbc
Aug 06 22:47:29 dist-test-slave-h3ft krb5kdc[3295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056449, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.193@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:47:29.760597 4300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:29.745051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:59073 (local address 127.18.45.254:43077)
0806 22:47:29.745214 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:29.745217 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:29.747545 (+ 2328us) server_negotiation.cc:407] Connection header received
0806 22:47:29.748468 (+ 923us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:29.748470 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:29.748493 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:29.748605 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:29.749632 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:29.750690 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:29.752383 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:29.752786 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:29.752873 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:29.756312 (+ 3439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:29.756314 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:29.756314 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:29.756324 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:29.758553 (+ 2229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:29.759221 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:29.759223 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:29.759224 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:29.759301 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:29.759660 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:29.759661 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:29.759662 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:29.759852 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:29.759924 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:29.760369 (+ 445us) server_negotiation.cc:299] Negotiation successful
0806 22:47:29.760466 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":84,"threads_started":1}
I20260806 22:47:29.760895 4299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:29.744998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:43077 (local address 127.18.45.193:59073)
0806 22:47:29.747346 (+ 2348us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:29.747407 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:29.748289 (+ 882us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:29.748738 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:29.748743 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:29.749135 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:29.749487 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:29.749494 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:29.750818 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:29.750819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:29.752271 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:29.752275 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:29.752913 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:29.752914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:29.753084 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:29.754149 (+ 1065us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:29.754162 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:29.756184 (+ 2022us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:29.758668 (+ 2484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:29.758677 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:29.758681 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:29.759119 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:29.759373 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:29.759373 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:29.759374 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:29.759543 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:29.759943 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:29.759951 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:29.760175 (+ 224us) client_negotiation.cc:769] Sending connection context
0806 22:47:29.760313 (+ 138us) client_negotiation.cc:240] Negotiation successful
0806 22:47:29.760526 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2293,"thread_start_us":114,"threads_started":1}
I20260806 22:47:29.761590 4296 heartbeater.cc:347] Connected to a master server at 127.18.45.254:43077
I20260806 22:47:29.761747 4296 heartbeater.cc:464] Registering TS with master...
I20260806 22:47:29.762003 4296 heartbeater.cc:511] Master 127.18.45.254:43077 requested a full tablet report, sending...
I20260806 22:47:29.762784 3993 ts_manager.cc:194] Registered new tserver with Master: eb8153447c6e45a2ab4d3360bae8c43a (127.18.45.193:40131)
I20260806 22:47:29.763902 3993 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.45.193@KRBTEST.COM'} at 127.18.45.193:59073
WARNING: no policy specified for kudu/127.18.45.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab.
Entry for principal kudu/127.18.45.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab.
I20260806 22:47:29.799566 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.45.212:55743/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.194:0
--local_ip_for_outbound_sockets=127.18.45.194
--webserver_interface=127.18.45.194
--webserver_port=0
--tserver_master_addrs=127.18.45.254:43077
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab
--principal=kudu/127.18.45.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:35547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 22:47:29 dist-test-slave-h3ft krb5kdc[3295](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786056449, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 22:47:29.939101 4303 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:29.939582 4303 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:29.939679 4303 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:29.941831 4303 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:29.941915 4303 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:29.941967 4303 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.194
I20260806 22:47:29.944522 4303 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:35547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.18.45.212:55743/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab
--principal=kudu/127.18.45.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.45.194
--webserver_port=0
--tserver_master_addrs=127.18.45.254:43077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4303
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.194
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:47:29.944868 4303 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:29.945199 4303 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:29.952678 4313 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:29.953004 4318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:29.952720 4312 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:29.953368 4303 server_base.cc:1034] running on GCE node
I20260806 22:47:29.953615 4303 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:29.953930 4303 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:29.955091 4303 hybrid_clock.cc:648] HybridClock initialized: now 1786056449955062 us; error 42 us; skew 500 ppm
Aug 06 22:47:29 dist-test-slave-h3ft krb5kdc[3295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056449, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:29.960242 4303 init.cc:377] Logged in from keytab as kudu/127.18.45.194@KRBTEST.COM (short username kudu)
I20260806 22:47:29.960947 4303 webserver.cc:460] Webserver started at http://127.18.45.194:44473/ using document root <none> and password file <none>
I20260806 22:47:29.961288 4303 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:29.961364 4303 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:29.961540 4303 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 22:47:29 dist-test-slave-h3ft krb5kdc[3295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056449, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.194@KRBTEST.COM for HTTP/127.18.45.212@KRBTEST.COM
I20260806 22:47:29.979926 4303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-1/data/instance:
uuid: "e36d326e506840fbab8cfec3b34d8366"
format_stamp: "Formatted at 2026-08-06 22:47:29 on dist-test-slave-h3ft"
server_key: "e9c8f1b5563bba3b0249e42603987546"
server_key_iv: "b69d302c361dcec3387b0df3de3f860c"
server_key_version: "kuduclusterkey@0"
I20260806 22:47:29.980535 4303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-1/wal/instance:
uuid: "e36d326e506840fbab8cfec3b34d8366"
format_stamp: "Formatted at 2026-08-06 22:47:29 on dist-test-slave-h3ft"
server_key: "e9c8f1b5563bba3b0249e42603987546"
server_key_iv: "b69d302c361dcec3387b0df3de3f860c"
server_key_version: "kuduclusterkey@0"
I20260806 22:47:29.985636 4303 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260806 22:47:30.015741 4332 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:30.016135 4303 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.000s sys 0.000s
I20260806 22:47:30.016278 4303 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-1/wal
uuid: "e36d326e506840fbab8cfec3b34d8366"
format_stamp: "Formatted at 2026-08-06 22:47:29 on dist-test-slave-h3ft"
server_key: "e9c8f1b5563bba3b0249e42603987546"
server_key_iv: "b69d302c361dcec3387b0df3de3f860c"
server_key_version: "kuduclusterkey@0"
I20260806 22:47:30.016422 4303 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:30.034497 4303 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:30.037547 4303 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:30.037765 4303 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:30.038152 4303 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:47:30.038643 4303 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:47:30.038702 4303 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:30.038749 4303 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:47:30.038775 4303 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:30.047819 4303 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.194:39045
I20260806 22:47:30.048084 4486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.194:39045 every 8 connection(s)
I20260806 22:47:30.048378 4303 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
I20260806 22:47:30.053131 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 4303
I20260806 22:47:30.053299 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-1/wal/instance
I20260806 22:47:30.053601 18615 external_mini_cluster.cc:1428] Setting key c3e2db9f7c1190112863ce0c29b25f6c
Aug 06 22:47:30 dist-test-slave-h3ft krb5kdc[3295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056449, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.194@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:47:30.064512 4300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:30.049376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:37141 (local address 127.18.45.254:43077)
0806 22:47:30.049721 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:30.049725 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:30.049770 (+ 45us) server_negotiation.cc:407] Connection header received
0806 22:47:30.050707 (+ 937us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:30.050710 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:30.050737 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:30.050776 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:30.051924 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:30.052815 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:30.055756 (+ 2941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:30.056103 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:30.056161 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:30.059654 (+ 3493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:30.059657 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:30.059657 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:30.059668 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:30.061837 (+ 2169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:30.062570 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:30.062572 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:30.062573 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:30.062658 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:30.063133 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:30.063135 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:30.063135 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:30.063324 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:30.063399 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:30.063891 (+ 492us) server_negotiation.cc:299] Negotiation successful
0806 22:47:30.064333 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299}
I20260806 22:47:30.065685 4491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:30.049373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:43077 (local address 127.18.45.194:37141)
0806 22:47:30.049607 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:30.049661 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:30.050533 (+ 872us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:30.051057 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:30.051064 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:30.051410 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:30.051774 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:30.051781 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:30.054069 (+ 2288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:30.054071 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:30.055627 (+ 1556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:30.055632 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:30.056311 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:30.056312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:30.056515 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:30.057297 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:30.057312 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:30.059504 (+ 2192us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:30.061985 (+ 2481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:30.062000 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:30.062004 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:30.062452 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:30.062742 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:30.062743 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:30.062743 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:30.063033 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:30.063442 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:30.063451 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:30.063694 (+ 243us) client_negotiation.cc:769] Sending connection context
0806 22:47:30.065089 (+ 1395us) client_negotiation.cc:240] Negotiation successful
0806 22:47:30.065359 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":125,"threads_started":1}
I20260806 22:47:30.066761 4487 heartbeater.cc:347] Connected to a master server at 127.18.45.254:43077
I20260806 22:47:30.066901 4487 heartbeater.cc:464] Registering TS with master...
I20260806 22:47:30.067245 4487 heartbeater.cc:511] Master 127.18.45.254:43077 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.18.45.195@KRBTEST.COM; defaulting to no policy
I20260806 22:47:30.067898 3993 ts_manager.cc:194] Registered new tserver with Master: e36d326e506840fbab8cfec3b34d8366 (127.18.45.194:39045)
I20260806 22:47:30.068468 3993 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.45.194@KRBTEST.COM'} at 127.18.45.194:37141
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab.
Entry for principal kudu/127.18.45.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab.
I20260806 22:47:30.101702 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.45.212:55743/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.195:0
--local_ip_for_outbound_sockets=127.18.45.195
--webserver_interface=127.18.45.195
--webserver_port=0
--tserver_master_addrs=127.18.45.254:43077
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab
--principal=kudu/127.18.45.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:35547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:47:30.224526 4494 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:30.224845 4494 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:30.224887 4494 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:30.226117 4494 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:30.226183 4494 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:30.226228 4494 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.195
I20260806 22:47:30.227952 4494 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:35547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.18.45.212:55743/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab
--principal=kudu/127.18.45.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.45.195
--webserver_port=0
--tserver_master_addrs=127.18.45.254:43077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4494
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.195
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:47:30.228255 4494 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:30.228600 4494 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:30.239862 4503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:30.239889 4504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:30.241578 4494 server_base.cc:1034] running on GCE node
W20260806 22:47:30.242138 4507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:30.242424 4494 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:30.242794 4494 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:30.243973 4494 hybrid_clock.cc:648] HybridClock initialized: now 1786056450243947 us; error 44 us; skew 500 ppm
Aug 06 22:47:30 dist-test-slave-h3ft krb5kdc[3295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056450, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:30.247999 4494 init.cc:377] Logged in from keytab as kudu/127.18.45.195@KRBTEST.COM (short username kudu)
I20260806 22:47:30.248690 4494 webserver.cc:460] Webserver started at http://127.18.45.195:40549/ using document root <none> and password file <none>
I20260806 22:47:30.249051 4494 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:30.249127 4494 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:30.249303 4494 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 22:47:30 dist-test-slave-h3ft krb5kdc[3295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056450, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.195@KRBTEST.COM for HTTP/127.18.45.212@KRBTEST.COM
I20260806 22:47:30.269075 4494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-2/data/instance:
uuid: "750754f0df8e44eeacc4b9a6a88794ae"
format_stamp: "Formatted at 2026-08-06 22:47:30 on dist-test-slave-h3ft"
server_key: "04545462807064eccc1e4f94e9b0af95"
server_key_iv: "9489c52b45d33b7d39666435530d9910"
server_key_version: "kuduclusterkey@0"
I20260806 22:47:30.269670 4494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-2/wal/instance:
uuid: "750754f0df8e44eeacc4b9a6a88794ae"
format_stamp: "Formatted at 2026-08-06 22:47:30 on dist-test-slave-h3ft"
server_key: "04545462807064eccc1e4f94e9b0af95"
server_key_iv: "9489c52b45d33b7d39666435530d9910"
server_key_version: "kuduclusterkey@0"
I20260806 22:47:30.280655 4494 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.005s sys 0.000s
I20260806 22:47:30.301620 4516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:30.301980 4494 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.000s
I20260806 22:47:30.302107 4494 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-2/wal
uuid: "750754f0df8e44eeacc4b9a6a88794ae"
format_stamp: "Formatted at 2026-08-06 22:47:30 on dist-test-slave-h3ft"
server_key: "04545462807064eccc1e4f94e9b0af95"
server_key_iv: "9489c52b45d33b7d39666435530d9910"
server_key_version: "kuduclusterkey@0"
I20260806 22:47:30.302239 4494 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:30.325241 4494 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:30.328079 4494 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:30.328284 4494 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:30.328557 4494 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:47:30.329336 4494 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:47:30.329399 4494 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:30.329435 4494 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:47:30.329461 4494 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:30.338267 4494 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.195:39247
I20260806 22:47:30.338780 4494 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
I20260806 22:47:30.344053 4670 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.195:39247 every 8 connection(s)
I20260806 22:47:30.346927 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 4494
I20260806 22:47:30.347098 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-2/wal/instance
I20260806 22:47:30.347316 18615 external_mini_cluster.cc:1428] Setting key 2e7e7e48aa5a4ec6e63465bec39a85bf
Aug 06 22:47:30 dist-test-slave-h3ft krb5kdc[3295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056450, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.195@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:47:30.361797 4300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:30.345277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.195:44137 (local address 127.18.45.254:43077)
0806 22:47:30.345336 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:30.345338 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:30.346200 (+ 862us) server_negotiation.cc:407] Connection header received
0806 22:47:30.347752 (+ 1552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:30.347753 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:30.347766 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:30.347822 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:30.349493 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:30.350338 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:30.351974 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:30.352296 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:30.352338 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:30.355501 (+ 3163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:30.355502 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:30.355503 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:30.355512 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:30.357432 (+ 1920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:30.358002 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:30.358003 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:30.358004 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:30.358071 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:30.358399 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:30.358400 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:30.358401 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:30.358575 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:30.358638 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:30.361573 (+ 2935us) server_negotiation.cc:299] Negotiation successful
0806 22:47:30.361680 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 22:47:30.362025 4673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:30.344884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:43077 (local address 127.18.45.195:44137)
0806 22:47:30.345990 (+ 1106us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:30.346052 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:30.346942 (+ 890us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:30.348723 (+ 1781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:30.348729 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:30.349030 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:30.349369 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:30.349375 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:30.350465 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:30.350466 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:30.351869 (+ 1403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:30.351873 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:30.352464 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:30.352465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:30.352661 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:30.353411 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:30.353424 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:30.355377 (+ 1953us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:30.357527 (+ 2150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:30.357536 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:30.357539 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:30.357907 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:30.358142 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:30.358143 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:30.358143 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:30.358309 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:30.358647 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:30.359349 (+ 702us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:30.359553 (+ 204us) client_negotiation.cc:769] Sending connection context
0806 22:47:30.361510 (+ 1957us) client_negotiation.cc:240] Negotiation successful
0806 22:47:30.361717 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1041,"thread_start_us":132,"threads_started":1}
I20260806 22:47:30.362762 4671 heartbeater.cc:347] Connected to a master server at 127.18.45.254:43077
I20260806 22:47:30.362870 4671 heartbeater.cc:464] Registering TS with master...
I20260806 22:47:30.363117 4671 heartbeater.cc:511] Master 127.18.45.254:43077 requested a full tablet report, sending...
I20260806 22:47:30.363654 3993 ts_manager.cc:194] Registered new tserver with Master: 750754f0df8e44eeacc4b9a6a88794ae (127.18.45.195:39247)
I20260806 22:47:30.364192 3993 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.45.195@KRBTEST.COM'} at 127.18.45.195:44137
I20260806 22:47:30.370299 18615 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:47:30.371227 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 4109
I20260806 22:47:30.439617 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 4303
Aug 06 22:47:30 dist-test-slave-h3ft krb5kdc[3295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056449, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.254@KRBTEST.COM for HTTP/127.18.45.212@KRBTEST.COM
I20260806 22:47:30.475641 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 4494
I20260806 22:47:30.487941 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 3942
I20260806 22:47:30.504747 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:43077
--webserver_interface=127.18.45.254
--webserver_port=41731
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:35547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.45.254:43077
--ranger_config_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.45.212:55743/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.18.45.212:55743/kms with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:47:30.604702 4674 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:30.604970 4674 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:30.604995 4674 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:30.606146 4674 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:47:30.606185 4674 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:30.606199 4674 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:30.606212 4674 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:47:30.606225 4674 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:47:30.607944 4674 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:35547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.18.45.212:55743/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.18.45.254:43077
--ranger_config_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:43077
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.45.254
--webserver_port=41731
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4674
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:47:30.608182 4674 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:30.608379 4674 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:30.613862 4686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:30.613865 4683 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:30.613900 4684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:30.614264 4674 server_base.cc:1034] running on GCE node
I20260806 22:47:30.614584 4674 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:30.614905 4674 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:30.616122 4674 hybrid_clock.cc:648] HybridClock initialized: now 1786056450616085 us; error 51 us; skew 500 ppm
Aug 06 22:47:30 dist-test-slave-h3ft krb5kdc[3295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056450, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:30.619107 4674 init.cc:377] Logged in from keytab as kudu/127.18.45.254@KRBTEST.COM (short username kudu)
I20260806 22:47:30.619568 4674 webserver.cc:460] Webserver started at http://127.18.45.254:41731/ using document root <none> and password file <none>
I20260806 22:47:30.619827 4674 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:30.619885 4674 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 22:47:30 dist-test-slave-h3ft krb5kdc[3295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056450, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.254@KRBTEST.COM for HTTP/127.18.45.212@KRBTEST.COM
I20260806 22:47:30.670226 4674 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260806 22:47:30.671307 4695 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:30.671553 4674 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:47:30.671707 4674 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/master-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/master-0/wal
uuid: "0c72db8fbc444b13bf1f88a9a4762d66"
format_stamp: "Formatted at 2026-08-06 22:47:28 on dist-test-slave-h3ft"
server_key: "0ad36734c764ac640e5bc4f049761032"
server_key_iv: "c7306fc02e56118ebcaf3525d8fe9397"
server_key_version: "kuduclusterkey@0"
I20260806 22:47:30.672199 4674 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:30.686389 4674 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:30.688990 4674 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:30.689213 4674 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:30.694741 4674 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.254:43077
I20260806 22:47:30.694780 4764 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.254:43077 every 8 connection(s)
I20260806 22:47:30.695330 4674 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
I20260806 22:47:30.696905 4765 sys_catalog.cc:263] Verifying existing consensus state
I20260806 22:47:30.697692 4765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c72db8fbc444b13bf1f88a9a4762d66: Bootstrap starting.
I20260806 22:47:30.703656 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 4674
I20260806 22:47:30.704124 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.45.212:55743/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.193:40131
--local_ip_for_outbound_sockets=127.18.45.193
--tserver_master_addrs=127.18.45.254:43077
--webserver_port=40581
--webserver_interface=127.18.45.193
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab
--principal=kudu/127.18.45.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:35547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.18.45.212:55743/kms with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:47:30.709167 4765 log.cc:824] T 00000000000000000000000000000000 P 0c72db8fbc444b13bf1f88a9a4762d66: Log is configured to *not* fsync() on all Append() calls
I20260806 22:47:30.711549 4765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c72db8fbc444b13bf1f88a9a4762d66: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:47:30.711912 4765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c72db8fbc444b13bf1f88a9a4762d66: Bootstrap complete.
I20260806 22:47:30.714985 4765 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c72db8fbc444b13bf1f88a9a4762d66 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c72db8fbc444b13bf1f88a9a4762d66" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 43077 } }
I20260806 22:47:30.715377 4765 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c72db8fbc444b13bf1f88a9a4762d66 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c72db8fbc444b13bf1f88a9a4762d66, State: Initialized, Role: FOLLOWER
I20260806 22:47:30.715590 4765 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c72db8fbc444b13bf1f88a9a4762d66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c72db8fbc444b13bf1f88a9a4762d66" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 43077 } }
I20260806 22:47:30.715699 4765 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0c72db8fbc444b13bf1f88a9a4762d66 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:47:30.715764 4765 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c72db8fbc444b13bf1f88a9a4762d66 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:47:30.715830 4765 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c72db8fbc444b13bf1f88a9a4762d66 [term 1 FOLLOWER]: Advancing to term 2
I20260806 22:47:30.717149 4765 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c72db8fbc444b13bf1f88a9a4762d66 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c72db8fbc444b13bf1f88a9a4762d66" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 43077 } }
I20260806 22:47:30.717257 4765 leader_election.cc:302] T 00000000000000000000000000000000 P 0c72db8fbc444b13bf1f88a9a4762d66 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c72db8fbc444b13bf1f88a9a4762d66; no voters:
I20260806 22:47:30.717511 4765 leader_election.cc:288] T 00000000000000000000000000000000 P 0c72db8fbc444b13bf1f88a9a4762d66 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 22:47:30.717551 4771 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0c72db8fbc444b13bf1f88a9a4762d66 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 22:47:30.717681 4771 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0c72db8fbc444b13bf1f88a9a4762d66 [term 2 LEADER]: Becoming Leader. State: Replica: 0c72db8fbc444b13bf1f88a9a4762d66, State: Running, Role: LEADER
I20260806 22:47:30.717862 4765 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c72db8fbc444b13bf1f88a9a4762d66 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:47:30.717842 4771 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c72db8fbc444b13bf1f88a9a4762d66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c72db8fbc444b13bf1f88a9a4762d66" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 43077 } }
I20260806 22:47:30.718308 4773 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c72db8fbc444b13bf1f88a9a4762d66 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c72db8fbc444b13bf1f88a9a4762d66. Latest consensus state: current_term: 2 leader_uuid: "0c72db8fbc444b13bf1f88a9a4762d66" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c72db8fbc444b13bf1f88a9a4762d66" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 43077 } } }
I20260806 22:47:30.718415 4773 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c72db8fbc444b13bf1f88a9a4762d66 [sys.catalog]: This master's current role is: LEADER
I20260806 22:47:30.718664 4772 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c72db8fbc444b13bf1f88a9a4762d66 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "0c72db8fbc444b13bf1f88a9a4762d66" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c72db8fbc444b13bf1f88a9a4762d66" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 43077 } } }
I20260806 22:47:30.718745 4772 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c72db8fbc444b13bf1f88a9a4762d66 [sys.catalog]: This master's current role is: LEADER
I20260806 22:47:30.719626 4765 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 22:47:30.720088 4781 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:47:30.723779 4781 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:47:30.724368 4781 catalog_manager.cc:1259] Loaded cluster ID: c577159029074be2ac8a0e2ffe316a44
I20260806 22:47:30.724494 4781 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:47:30.726372 4781 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:47:30.726697 4781 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0c72db8fbc444b13bf1f88a9a4762d66: Loaded TSK: 0
I20260806 22:47:30.727098 4781 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 22:47:30.837548 4768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:30.837965 4768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:30.838100 4768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:30.840006 4768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:30.840077 4768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:30.840122 4768 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.193
I20260806 22:47:30.842423 4768 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:35547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.18.45.212:55743/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab
--principal=kudu/127.18.45.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.193:40131
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.45.193
--webserver_port=40581
--tserver_master_addrs=127.18.45.254:43077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4768
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.193
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:47:30.842878 4768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:30.843323 4768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:30.851336 4818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:30.851804 4817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:30.852172 4820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:30.853430 4768 server_base.cc:1034] running on GCE node
I20260806 22:47:30.853782 4768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:30.854161 4768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:30.857445 4768 hybrid_clock.cc:648] HybridClock initialized: now 1786056450857405 us; error 47 us; skew 500 ppm
Aug 06 22:47:30 dist-test-slave-h3ft krb5kdc[3295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056450, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:30.861632 4768 init.cc:377] Logged in from keytab as kudu/127.18.45.193@KRBTEST.COM (short username kudu)
I20260806 22:47:30.862433 4768 webserver.cc:460] Webserver started at http://127.18.45.193:40581/ using document root <none> and password file <none>
I20260806 22:47:30.862809 4768 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:30.862921 4768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 22:47:30 dist-test-slave-h3ft krb5kdc[3295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056450, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.193@KRBTEST.COM for HTTP/127.18.45.212@KRBTEST.COM
I20260806 22:47:30.892390 4768 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:47:30.894042 4830 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:30.894389 4768 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 22:47:30.894524 4768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-0/wal
uuid: "eb8153447c6e45a2ab4d3360bae8c43a"
format_stamp: "Formatted at 2026-08-06 22:47:29 on dist-test-slave-h3ft"
server_key: "c40b8fb13f41c19e3088d5a86d8a1196"
server_key_iv: "dcae938ce40c963ba1b6ec56f0d930c0"
server_key_version: "kuduclusterkey@0"
I20260806 22:47:30.895135 4768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:30.927171 4768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:30.930157 4768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:30.930415 4768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:30.930774 4768 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:47:30.931273 4768 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:47:30.931363 4768 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:30.931484 4768 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:47:30.931568 4768 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:30.945861 4768 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.193:40131
I20260806 22:47:30.946431 4768 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
I20260806 22:47:30.946712 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 4768
I20260806 22:47:30.947131 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.45.212:55743/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.194:39045
--local_ip_for_outbound_sockets=127.18.45.194
--tserver_master_addrs=127.18.45.254:43077
--webserver_port=44473
--webserver_interface=127.18.45.194
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab
--principal=kudu/127.18.45.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:35547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.18.45.212:55743/kms with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:47:30.947104 4981 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.193:40131 every 8 connection(s)
Aug 06 22:47:30 dist-test-slave-h3ft krb5kdc[3295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056450, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.193@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:47:30.973676 4987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:30.948105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:43077 (local address 127.18.45.193:40401)
0806 22:47:30.949177 (+ 1072us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:30.949244 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:30.950201 (+ 957us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:30.950965 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:30.950971 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:30.951327 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:30.951702 (+ 375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:30.951710 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:30.953465 (+ 1755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:30.953466 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:30.954983 (+ 1517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:30.954986 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:30.956186 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:30.956188 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:30.956353 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:30.957129 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:30.957143 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:30.960328 (+ 3185us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:30.970366 (+ 10038us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:30.970384 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:30.970388 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:30.970817 (+ 429us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:30.971849 (+ 1032us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:30.971851 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:30.971851 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:30.972062 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:30.972774 (+ 712us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:30.972782 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:30.972984 (+ 202us) client_negotiation.cc:769] Sending connection context
0806 22:47:30.973121 (+ 137us) client_negotiation.cc:240] Negotiation successful
0806 22:47:30.973351 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1001,"thread_start_us":98,"threads_started":1}
I20260806 22:47:30.974740 4988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:30.948893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:40401 (local address 127.18.45.254:43077)
0806 22:47:30.949718 (+ 825us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:30.949729 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:30.949787 (+ 58us) server_negotiation.cc:407] Connection header received
0806 22:47:30.950560 (+ 773us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:30.950568 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:30.950641 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:30.950785 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:30.951847 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:30.953322 (+ 1475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:30.955129 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:30.955879 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:30.956005 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:30.960506 (+ 4501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:30.960513 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:30.960514 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:30.960530 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:30.970196 (+ 9666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:30.971227 (+ 1031us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:30.971233 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:30.971233 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:30.971397 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:30.972162 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:30.972163 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:30.972163 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:30.972640 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:30.972739 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:30.974323 (+ 1584us) server_negotiation.cc:299] Negotiation successful
0806 22:47:30.974493 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":752,"thread_start_us":98,"threads_started":1}
I20260806 22:47:30.975968 4983 heartbeater.cc:347] Connected to a master server at 127.18.45.254:43077
I20260806 22:47:30.976125 4983 heartbeater.cc:464] Registering TS with master...
I20260806 22:47:30.976389 4983 heartbeater.cc:511] Master 127.18.45.254:43077 requested a full tablet report, sending...
W20260806 22:47:30.977474 4983 heartbeater.cc:644] Failed to heartbeat to 127.18.45.254:43077 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 22:47:30.977559 4983 heartbeater.cc:464] Registering TS with master...
I20260806 22:47:30.977638 4983 heartbeater.cc:511] Master 127.18.45.254:43077 requested a full tablet report, sending...
I20260806 22:47:30.978158 4983 heartbeater.cc:464] Registering TS with master...
I20260806 22:47:30.978245 4983 heartbeater.cc:511] Master 127.18.45.254:43077 requested a full tablet report, sending...
W20260806 22:47:30.978626 4983 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 22:47:31.082259 4989 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:31.082544 4989 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:31.082571 4989 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:31.084317 4989 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:31.084443 4989 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:31.084519 4989 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.194
I20260806 22:47:31.087343 4989 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:35547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.18.45.212:55743/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab
--principal=kudu/127.18.45.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.194:39045
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.45.194
--webserver_port=44473
--tserver_master_addrs=127.18.45.254:43077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4989
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.194
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:47:31.087812 4989 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:31.088240 4989 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:31.095646 4996 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:31.095671 4994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:31.096246 4989 server_base.cc:1034] running on GCE node
W20260806 22:47:31.096342 5000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:31.096848 4989 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:31.097270 4989 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:31.104671 4989 hybrid_clock.cc:648] HybridClock initialized: now 1786056451104638 us; error 48 us; skew 500 ppm
Aug 06 22:47:31 dist-test-slave-h3ft krb5kdc[3295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056451, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:31.108994 4989 init.cc:377] Logged in from keytab as kudu/127.18.45.194@KRBTEST.COM (short username kudu)
I20260806 22:47:31.109787 4989 webserver.cc:460] Webserver started at http://127.18.45.194:44473/ using document root <none> and password file <none>
I20260806 22:47:31.110082 4989 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:31.110147 4989 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 22:47:31 dist-test-slave-h3ft krb5kdc[3295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056451, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.194@KRBTEST.COM for HTTP/127.18.45.212@KRBTEST.COM
I20260806 22:47:31.139539 4989 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.002s sys 0.000s
I20260806 22:47:31.140938 5009 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:31.141191 4989 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:47:31.141314 4989 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-1/wal
uuid: "e36d326e506840fbab8cfec3b34d8366"
format_stamp: "Formatted at 2026-08-06 22:47:29 on dist-test-slave-h3ft"
server_key: "e9c8f1b5563bba3b0249e42603987546"
server_key_iv: "b69d302c361dcec3387b0df3de3f860c"
server_key_version: "kuduclusterkey@0"
I20260806 22:47:31.141849 4989 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:31.202433 4989 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:31.205498 4989 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:31.205704 4989 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:31.206049 4989 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:47:31.206512 4989 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:47:31.206576 4989 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:31.206616 4989 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:47:31.206678 4989 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:31.214438 4989 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.194:39045
I20260806 22:47:31.215191 4989 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
I20260806 22:47:31.219231 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 4989
I20260806 22:47:31.219591 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.45.212:55743/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.195:39247
--local_ip_for_outbound_sockets=127.18.45.195
--tserver_master_addrs=127.18.45.254:43077
--webserver_port=40549
--webserver_interface=127.18.45.195
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab
--principal=kudu/127.18.45.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:35547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.18.45.212:55743/kms with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:47:31.218132 5175 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.194:39045 every 8 connection(s)
Aug 06 22:47:31 dist-test-slave-h3ft krb5kdc[3295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056451, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.194@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:47:31.234956 4988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:31.218021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:33107 (local address 127.18.45.254:43077)
0806 22:47:31.218097 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:31.218100 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:31.220670 (+ 2570us) server_negotiation.cc:407] Connection header received
0806 22:47:31.221384 (+ 714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:31.221386 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:31.221410 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:31.221498 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:31.222425 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:31.223277 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:31.224979 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:31.225314 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:31.225363 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:31.230966 (+ 5603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:31.230969 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:31.230969 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:31.230980 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:31.233055 (+ 2075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:31.233679 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:31.233681 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:31.233682 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:31.233753 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:31.234049 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:31.234050 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:31.234050 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:31.234216 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:31.234263 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:31.234709 (+ 446us) server_negotiation.cc:299] Negotiation successful
0806 22:47:31.234825 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 22:47:31.236001 5178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:31.217651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:43077 (local address 127.18.45.194:33107)
0806 22:47:31.220450 (+ 2799us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:31.220525 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:31.221260 (+ 735us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:31.221626 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:31.221632 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:31.221941 (+ 309us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:31.222296 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:31.222304 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:31.223408 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:31.223410 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:31.224857 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:31.224861 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:31.225503 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:31.225504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:31.225681 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:31.228060 (+ 2379us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:31.228076 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:31.230811 (+ 2735us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:31.233171 (+ 2360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:31.233182 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:31.233186 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:31.233578 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:31.233825 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:31.233826 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:31.233826 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:31.233978 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:31.234311 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:31.234317 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:31.234526 (+ 209us) client_negotiation.cc:769] Sending connection context
0806 22:47:31.235451 (+ 925us) client_negotiation.cc:240] Negotiation successful
0806 22:47:31.235699 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2729,"thread_start_us":105,"threads_started":1}
I20260806 22:47:31.236763 5176 heartbeater.cc:347] Connected to a master server at 127.18.45.254:43077
I20260806 22:47:31.236934 5176 heartbeater.cc:464] Registering TS with master...
I20260806 22:47:31.237238 5176 heartbeater.cc:511] Master 127.18.45.254:43077 requested a full tablet report, sending...
W20260806 22:47:31.238075 5176 heartbeater.cc:644] Failed to heartbeat to 127.18.45.254:43077 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 22:47:31.238220 5176 heartbeater.cc:464] Registering TS with master...
I20260806 22:47:31.238341 5176 heartbeater.cc:511] Master 127.18.45.254:43077 requested a full tablet report, sending...
I20260806 22:47:31.238732 5176 heartbeater.cc:464] Registering TS with master...
I20260806 22:47:31.238827 5176 heartbeater.cc:511] Master 127.18.45.254:43077 requested a full tablet report, sending...
W20260806 22:47:31.239197 5176 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 22:47:31.362843 5179 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:31.363268 5179 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:31.363313 5179 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:31.365123 5179 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:31.365193 5179 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:31.365226 5179 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.195
I20260806 22:47:31.367257 5179 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:35547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.18.45.212:55743/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab
--principal=kudu/127.18.45.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.195:39247
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.45.195
--webserver_port=40549
--tserver_master_addrs=127.18.45.254:43077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5179
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.195
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:47:31.367596 5179 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:31.367976 5179 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:31.377475 5190 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:31.377688 5179 server_base.cc:1034] running on GCE node
W20260806 22:47:31.377453 5193 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:31.379159 5189 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:31.379473 5179 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:31.379841 5179 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:31.381059 5179 hybrid_clock.cc:648] HybridClock initialized: now 1786056451381024 us; error 51 us; skew 500 ppm
Aug 06 22:47:31 dist-test-slave-h3ft krb5kdc[3295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056451, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:31.384708 5179 init.cc:377] Logged in from keytab as kudu/127.18.45.195@KRBTEST.COM (short username kudu)
I20260806 22:47:31.385329 5179 webserver.cc:460] Webserver started at http://127.18.45.195:40549/ using document root <none> and password file <none>
I20260806 22:47:31.385715 5179 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:31.385802 5179 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 22:47:31 dist-test-slave-h3ft krb5kdc[3295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056451, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.195@KRBTEST.COM for HTTP/127.18.45.212@KRBTEST.COM
I20260806 22:47:31.407893 5179 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260806 22:47:31.409312 5203 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:31.409658 5179 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:47:31.409794 5179 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-2/wal
uuid: "750754f0df8e44eeacc4b9a6a88794ae"
format_stamp: "Formatted at 2026-08-06 22:47:30 on dist-test-slave-h3ft"
server_key: "04545462807064eccc1e4f94e9b0af95"
server_key_iv: "9489c52b45d33b7d39666435530d9910"
server_key_version: "kuduclusterkey@0"
I20260806 22:47:31.410411 5179 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:31.432992 5179 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:31.435858 5179 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:31.436057 5179 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:31.436391 5179 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:47:31.436901 5179 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:47:31.437016 5179 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:31.437115 5179 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:47:31.437180 5179 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:31.445937 5179 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.195:39247
I20260806 22:47:31.446626 5179 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
I20260806 22:47:31.447927 5357 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.195:39247 every 8 connection(s)
I20260806 22:47:31.451603 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 5179
Aug 06 22:47:31 dist-test-slave-h3ft krb5kdc[3295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056451, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.195@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:47:31.463961 4988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:31.447697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.195:45565 (local address 127.18.45.254:43077)
0806 22:47:31.447765 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:31.447768 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:31.451402 (+ 3634us) server_negotiation.cc:407] Connection header received
0806 22:47:31.452294 (+ 892us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:31.452296 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:31.452313 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:31.452350 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:31.453412 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:31.454302 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:31.456001 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:31.456290 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:31.456336 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:31.460132 (+ 3796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:31.460134 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:31.460135 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:31.460143 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:31.462074 (+ 1931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:31.462691 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:31.462693 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:31.462693 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:31.462765 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:31.463173 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:31.463174 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:31.463174 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:31.463336 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:31.463402 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:31.463749 (+ 347us) server_negotiation.cc:299] Negotiation successful
0806 22:47:31.463850 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 22:47:31.464895 5362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:31.447722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:43077 (local address 127.18.45.195:45565)
0806 22:47:31.451185 (+ 3463us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:31.451254 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:31.452140 (+ 886us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:31.452637 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:31.452643 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:31.452932 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:31.453283 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:31.453290 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:31.454451 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:31.454452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:31.455894 (+ 1442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:31.455897 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:31.456472 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:31.456473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:31.456640 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:31.458296 (+ 1656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:31.458308 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:31.460005 (+ 1697us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:31.462179 (+ 2174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:31.462188 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:31.462191 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:31.462563 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:31.462839 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:31.462840 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:31.462840 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:31.463044 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:31.463407 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:31.463416 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:31.463592 (+ 176us) client_negotiation.cc:769] Sending connection context
0806 22:47:31.463712 (+ 120us) client_negotiation.cc:240] Negotiation successful
0806 22:47:31.463893 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3395,"thread_start_us":119,"threads_started":1}
I20260806 22:47:31.465482 5359 heartbeater.cc:347] Connected to a master server at 127.18.45.254:43077
I20260806 22:47:31.465632 5359 heartbeater.cc:464] Registering TS with master...
I20260806 22:47:31.465865 5359 heartbeater.cc:511] Master 127.18.45.254:43077 requested a full tablet report, sending...
W20260806 22:47:31.466739 5359 heartbeater.cc:644] Failed to heartbeat to 127.18.45.254:43077 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 22:47:31.466890 5359 heartbeater.cc:464] Registering TS with master...
I20260806 22:47:31.467044 5359 heartbeater.cc:511] Master 127.18.45.254:43077 requested a full tablet report, sending...
I20260806 22:47:31.467249 5364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:31.452135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60360 (local address 127.18.45.254:43077)
0806 22:47:31.457029 (+ 4894us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:31.457032 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:31.457233 (+ 201us) server_negotiation.cc:407] Connection header received
0806 22:47:31.457339 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:31.457340 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:31.457357 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:31.457461 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:31.458162 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:31.458873 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:31.460216 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:31.460525 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:31.460609 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:31.462188 (+ 1579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:31.462189 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:31.462189 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:31.462196 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:31.464608 (+ 2412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:31.465189 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:31.465190 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:31.465191 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:31.465351 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:31.466320 (+ 969us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:31.466322 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:31.466322 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:31.466528 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:31.466753 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:31.466793 (+ 40us) server_negotiation.cc:299] Negotiation successful
0806 22:47:31.467123 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4864,"thread_start_us":84,"threads_started":1}
I20260806 22:47:31.467541 5359 heartbeater.cc:464] Registering TS with master...
I20260806 22:47:31.467630 5359 heartbeater.cc:511] Master 127.18.45.254:43077 requested a full tablet report, sending...
W20260806 22:47:31.467939 5359 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 22:47:31.978849 4983 heartbeater.cc:464] Registering TS with master...
I20260806 22:47:31.979123 4983 heartbeater.cc:511] Master 127.18.45.254:43077 requested a full tablet report, sending...
I20260806 22:47:31.979717 4715 ts_manager.cc:194] Registered new tserver with Master: eb8153447c6e45a2ab4d3360bae8c43a (127.18.45.193:40131)
I20260806 22:47:31.980929 4715 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.45.193@KRBTEST.COM'} at 127.18.45.193:40401
Aug 06 22:47:31 dist-test-slave-h3ft krb5kdc[3295](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786056451, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:32.239450 5176 heartbeater.cc:464] Registering TS with master...
I20260806 22:47:32.239682 5176 heartbeater.cc:511] Master 127.18.45.254:43077 requested a full tablet report, sending...
I20260806 22:47:32.240099 4715 ts_manager.cc:194] Registered new tserver with Master: e36d326e506840fbab8cfec3b34d8366 (127.18.45.194:39045)
I20260806 22:47:32.240743 4715 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.45.194@KRBTEST.COM'} at 127.18.45.194:33107
Aug 06 22:47:32 dist-test-slave-h3ft krb5kdc[3295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056451, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.254@KRBTEST.COM for HTTP/127.18.45.212@KRBTEST.COM
I20260806 22:47:32.468204 5359 heartbeater.cc:464] Registering TS with master...
I20260806 22:47:32.468447 5359 heartbeater.cc:511] Master 127.18.45.254:43077 requested a full tablet report, sending...
I20260806 22:47:32.468839 4715 ts_manager.cc:194] Registered new tserver with Master: 750754f0df8e44eeacc4b9a6a88794ae (127.18.45.195:39247)
I20260806 22:47:32.469340 4715 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.45.195@KRBTEST.COM'} at 127.18.45.195:45565
I20260806 22:47:32.478466 18615 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:47:32.499171 5388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:32.480713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60370 (local address 127.18.45.254:43077)
0806 22:47:32.488720 (+ 8007us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:32.488724 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:32.488743 (+ 19us) server_negotiation.cc:407] Connection header received
0806 22:47:32.488786 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:32.488787 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:32.488809 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:32.488954 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:32.490015 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:32.490974 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:32.492466 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:32.492862 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:32.492947 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:32.494961 (+ 2014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:32.494964 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:32.494964 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:32.494976 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:32.497252 (+ 2276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:32.497736 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:32.497737 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:32.497738 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:32.497805 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:32.498033 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:32.498034 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:32.498034 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:32.498197 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:32.498312 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:32.498908 (+ 596us) server_negotiation.cc:299] Negotiation successful
0806 22:47:32.499038 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7960,"thread_start_us":88,"threads_started":1}
I20260806 22:47:32.502532 4715 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60370:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:47:32.506268 4715 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:47:32.517310 5396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:32.510432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.194:39045 (local address 127.0.0.1:54966)
0806 22:47:32.512147 (+ 1715us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:32.512157 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:32.512204 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:32.512494 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:32.512495 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:32.512497 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:47:32.512597 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:32.512599 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:32.513997 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:32.513998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:32.515484 (+ 1486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:32.515487 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:32.516927 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:32.516929 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:32.517039 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:32.517055 (+ 16us) client_negotiation.cc:769] Sending connection context
0806 22:47:32.517129 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 22:47:32.517201 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1699,"mutex_wait_us":175,"thread_start_us":110,"threads_started":1}
I20260806 22:47:32.517380 5403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:32.510985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41322 (local address 127.18.45.195:39247)
0806 22:47:32.511865 (+ 880us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:32.511870 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:32.511880 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:47:32.511923 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:32.511925 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:32.511942 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:32.512022 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:47:32.512276 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:32.513806 (+ 1530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:32.515651 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:32.516648 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:32.516733 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:32.516820 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:32.517198 (+ 378us) server_negotiation.cc:299] Negotiation successful
0806 22:47:32.517256 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":847,"thread_start_us":321,"threads_started":1}
I20260806 22:47:32.517935 5399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:32.510431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.195:39247 (local address 127.0.0.1:41322)
0806 22:47:32.511607 (+ 1176us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:32.511656 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:32.511733 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:32.512062 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:32.512066 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:32.512069 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:47:32.512166 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:32.512168 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:32.513930 (+ 1762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:32.513932 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:32.515535 (+ 1603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:32.515538 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:32.516929 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:32.516930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:32.517043 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:32.517056 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 22:47:32.517139 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 22:47:32.517861 (+ 722us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1143,"mutex_wait_us":670,"thread_start_us":121,"threads_started":1}
I20260806 22:47:32.518214 5402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:32.511096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54966 (local address 127.18.45.194:39045)
0806 22:47:32.511436 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:32.511443 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:32.512257 (+ 814us) server_negotiation.cc:407] Connection header received
0806 22:47:32.512299 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:32.512301 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:32.512317 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:32.512394 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:47:32.512670 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:32.513907 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:32.515581 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:32.516763 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:32.516833 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:32.516891 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:32.518032 (+ 1141us) server_negotiation.cc:299] Negotiation successful
0806 22:47:32.518127 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":83,"threads_started":1}
I20260806 22:47:32.518988 5272 tablet_service.cc:1467] Processing CreateTablet for tablet b84b761b96784f04826f21bf0828eb5f (DEFAULT_TABLE table=test-table [id=1ec22b6c5e9842b781911044cbf13cf1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:47:32.519201 5077 tablet_service.cc:1467] Processing CreateTablet for tablet b84b761b96784f04826f21bf0828eb5f (DEFAULT_TABLE table=test-table [id=1ec22b6c5e9842b781911044cbf13cf1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:47:32.519556 5272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b84b761b96784f04826f21bf0828eb5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:47:32.519615 5077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b84b761b96784f04826f21bf0828eb5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:47:32.527242 5404 tablet_bootstrap.cc:492] T b84b761b96784f04826f21bf0828eb5f P 750754f0df8e44eeacc4b9a6a88794ae: Bootstrap starting.
I20260806 22:47:32.528239 5404 tablet_bootstrap.cc:654] T b84b761b96784f04826f21bf0828eb5f P 750754f0df8e44eeacc4b9a6a88794ae: Neither blocks nor log segments found. Creating new log.
I20260806 22:47:32.528630 5404 log.cc:824] T b84b761b96784f04826f21bf0828eb5f P 750754f0df8e44eeacc4b9a6a88794ae: Log is configured to *not* fsync() on all Append() calls
I20260806 22:47:32.529140 5407 tablet_bootstrap.cc:492] T b84b761b96784f04826f21bf0828eb5f P e36d326e506840fbab8cfec3b34d8366: Bootstrap starting.
I20260806 22:47:32.529858 5398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:32.510709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.193:40131 (local address 127.0.0.1:59830)
0806 22:47:32.514208 (+ 3499us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:32.514218 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:32.514269 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:32.522473 (+ 8204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:32.522475 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:32.522478 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:47:32.522544 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:32.522546 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:32.527112 (+ 4566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:32.527113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:32.528264 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:32.528268 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:32.529580 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:32.529582 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:32.529653 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:32.529661 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 22:47:32.529691 (+ 30us) client_negotiation.cc:240] Negotiation successful
0806 22:47:32.529745 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3473,"thread_start_us":84,"threads_started":1}
I20260806 22:47:32.530031 5407 tablet_bootstrap.cc:654] T b84b761b96784f04826f21bf0828eb5f P e36d326e506840fbab8cfec3b34d8366: Neither blocks nor log segments found. Creating new log.
I20260806 22:47:32.530454 5407 log.cc:824] T b84b761b96784f04826f21bf0828eb5f P e36d326e506840fbab8cfec3b34d8366: Log is configured to *not* fsync() on all Append() calls
I20260806 22:47:32.530602 5400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:32.510864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59830 (local address 127.18.45.193:40131)
0806 22:47:32.521307 (+ 10443us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:32.521314 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:32.521341 (+ 27us) server_negotiation.cc:407] Connection header received
0806 22:47:32.521388 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:32.521391 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:32.521412 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:32.522646 (+ 1234us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:47:32.522763 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:32.523881 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:32.528499 (+ 4618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:32.529471 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:32.530304 (+ 833us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:32.530398 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:32.530429 (+ 31us) server_negotiation.cc:299] Negotiation successful
0806 22:47:32.530480 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10385,"thread_start_us":89,"threads_started":1}
I20260806 22:47:32.531623 5404 tablet_bootstrap.cc:492] T b84b761b96784f04826f21bf0828eb5f P 750754f0df8e44eeacc4b9a6a88794ae: No bootstrap required, opened a new log
I20260806 22:47:32.531569 4889 tablet_service.cc:1467] Processing CreateTablet for tablet b84b761b96784f04826f21bf0828eb5f (DEFAULT_TABLE table=test-table [id=1ec22b6c5e9842b781911044cbf13cf1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:47:32.531742 5404 ts_tablet_manager.cc:1397] T b84b761b96784f04826f21bf0828eb5f P 750754f0df8e44eeacc4b9a6a88794ae: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260806 22:47:32.532004 4889 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b84b761b96784f04826f21bf0828eb5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:47:32.534663 5404 raft_consensus.cc:348] T b84b761b96784f04826f21bf0828eb5f P 750754f0df8e44eeacc4b9a6a88794ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "750754f0df8e44eeacc4b9a6a88794ae" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 39247 } } peers { permanent_uuid: "e36d326e506840fbab8cfec3b34d8366" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 39045 } } peers { permanent_uuid: "eb8153447c6e45a2ab4d3360bae8c43a" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 40131 } }
I20260806 22:47:32.534876 5404 raft_consensus.cc:374] T b84b761b96784f04826f21bf0828eb5f P 750754f0df8e44eeacc4b9a6a88794ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:47:32.534915 5404 raft_consensus.cc:729] T b84b761b96784f04826f21bf0828eb5f P 750754f0df8e44eeacc4b9a6a88794ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 750754f0df8e44eeacc4b9a6a88794ae, State: Initialized, Role: FOLLOWER
I20260806 22:47:32.535180 5404 consensus_queue.cc:260] T b84b761b96784f04826f21bf0828eb5f P 750754f0df8e44eeacc4b9a6a88794ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "750754f0df8e44eeacc4b9a6a88794ae" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 39247 } } peers { permanent_uuid: "e36d326e506840fbab8cfec3b34d8366" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 39045 } } peers { permanent_uuid: "eb8153447c6e45a2ab4d3360bae8c43a" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 40131 } }
I20260806 22:47:32.535523 5404 ts_tablet_manager.cc:1428] T b84b761b96784f04826f21bf0828eb5f P 750754f0df8e44eeacc4b9a6a88794ae: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260806 22:47:32.535679 5359 heartbeater.cc:503] Master 127.18.45.254:43077 was elected leader, sending a full tablet report...
I20260806 22:47:32.536417 5407 tablet_bootstrap.cc:492] T b84b761b96784f04826f21bf0828eb5f P e36d326e506840fbab8cfec3b34d8366: No bootstrap required, opened a new log
I20260806 22:47:32.536512 5407 ts_tablet_manager.cc:1397] T b84b761b96784f04826f21bf0828eb5f P e36d326e506840fbab8cfec3b34d8366: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260806 22:47:32.539808 5407 raft_consensus.cc:348] T b84b761b96784f04826f21bf0828eb5f P e36d326e506840fbab8cfec3b34d8366 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "750754f0df8e44eeacc4b9a6a88794ae" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 39247 } } peers { permanent_uuid: "e36d326e506840fbab8cfec3b34d8366" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 39045 } } peers { permanent_uuid: "eb8153447c6e45a2ab4d3360bae8c43a" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 40131 } }
I20260806 22:47:32.540050 5407 raft_consensus.cc:374] T b84b761b96784f04826f21bf0828eb5f P e36d326e506840fbab8cfec3b34d8366 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:47:32.540088 5407 raft_consensus.cc:729] T b84b761b96784f04826f21bf0828eb5f P e36d326e506840fbab8cfec3b34d8366 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e36d326e506840fbab8cfec3b34d8366, State: Initialized, Role: FOLLOWER
I20260806 22:47:32.540292 5407 consensus_queue.cc:260] T b84b761b96784f04826f21bf0828eb5f P e36d326e506840fbab8cfec3b34d8366 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "750754f0df8e44eeacc4b9a6a88794ae" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 39247 } } peers { permanent_uuid: "e36d326e506840fbab8cfec3b34d8366" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 39045 } } peers { permanent_uuid: "eb8153447c6e45a2ab4d3360bae8c43a" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 40131 } }
I20260806 22:47:32.540637 5407 ts_tablet_manager.cc:1428] T b84b761b96784f04826f21bf0828eb5f P e36d326e506840fbab8cfec3b34d8366: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260806 22:47:32.540812 5176 heartbeater.cc:503] Master 127.18.45.254:43077 was elected leader, sending a full tablet report...
I20260806 22:47:32.541687 5418 tablet_bootstrap.cc:492] T b84b761b96784f04826f21bf0828eb5f P eb8153447c6e45a2ab4d3360bae8c43a: Bootstrap starting.
I20260806 22:47:32.542497 5418 tablet_bootstrap.cc:654] T b84b761b96784f04826f21bf0828eb5f P eb8153447c6e45a2ab4d3360bae8c43a: Neither blocks nor log segments found. Creating new log.
I20260806 22:47:32.542829 5418 log.cc:824] T b84b761b96784f04826f21bf0828eb5f P eb8153447c6e45a2ab4d3360bae8c43a: Log is configured to *not* fsync() on all Append() calls
I20260806 22:47:32.543869 5418 tablet_bootstrap.cc:492] T b84b761b96784f04826f21bf0828eb5f P eb8153447c6e45a2ab4d3360bae8c43a: No bootstrap required, opened a new log
I20260806 22:47:32.543956 5418 ts_tablet_manager.cc:1397] T b84b761b96784f04826f21bf0828eb5f P eb8153447c6e45a2ab4d3360bae8c43a: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260806 22:47:32.546831 5418 raft_consensus.cc:348] T b84b761b96784f04826f21bf0828eb5f P eb8153447c6e45a2ab4d3360bae8c43a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "750754f0df8e44eeacc4b9a6a88794ae" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 39247 } } peers { permanent_uuid: "e36d326e506840fbab8cfec3b34d8366" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 39045 } } peers { permanent_uuid: "eb8153447c6e45a2ab4d3360bae8c43a" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 40131 } }
I20260806 22:47:32.547091 5418 raft_consensus.cc:374] T b84b761b96784f04826f21bf0828eb5f P eb8153447c6e45a2ab4d3360bae8c43a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:47:32.547171 5418 raft_consensus.cc:729] T b84b761b96784f04826f21bf0828eb5f P eb8153447c6e45a2ab4d3360bae8c43a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb8153447c6e45a2ab4d3360bae8c43a, State: Initialized, Role: FOLLOWER
I20260806 22:47:32.547343 5418 consensus_queue.cc:260] T b84b761b96784f04826f21bf0828eb5f P eb8153447c6e45a2ab4d3360bae8c43a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "750754f0df8e44eeacc4b9a6a88794ae" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 39247 } } peers { permanent_uuid: "e36d326e506840fbab8cfec3b34d8366" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 39045 } } peers { permanent_uuid: "eb8153447c6e45a2ab4d3360bae8c43a" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 40131 } }
I20260806 22:47:32.547725 4983 heartbeater.cc:503] Master 127.18.45.254:43077 was elected leader, sending a full tablet report...
I20260806 22:47:32.547727 5418 ts_tablet_manager.cc:1428] T b84b761b96784f04826f21bf0828eb5f P eb8153447c6e45a2ab4d3360bae8c43a: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260806 22:47:32.685199 5419 raft_consensus.cc:482] T b84b761b96784f04826f21bf0828eb5f P e36d326e506840fbab8cfec3b34d8366 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:47:32.685433 5419 raft_consensus.cc:504] T b84b761b96784f04826f21bf0828eb5f P e36d326e506840fbab8cfec3b34d8366 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "750754f0df8e44eeacc4b9a6a88794ae" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 39247 } } peers { permanent_uuid: "e36d326e506840fbab8cfec3b34d8366" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 39045 } } peers { permanent_uuid: "eb8153447c6e45a2ab4d3360bae8c43a" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 40131 } }
I20260806 22:47:32.686004 5419 leader_election.cc:288] T b84b761b96784f04826f21bf0828eb5f P e36d326e506840fbab8cfec3b34d8366 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 750754f0df8e44eeacc4b9a6a88794ae (127.18.45.195:39247), eb8153447c6e45a2ab4d3360bae8c43a (127.18.45.193:40131)
I20260806 22:47:32.690975 5403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:32.686643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:46953 (local address 127.18.45.195:39247)
0806 22:47:32.686705 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:32.686708 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:32.686907 (+ 199us) server_negotiation.cc:407] Connection header received
0806 22:47:32.687081 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:32.687083 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:32.687095 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:32.687129 (+ 34us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:47:32.687451 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:32.688255 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:32.689885 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:32.690505 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:32.690558 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:32.690568 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:32.690817 (+ 249us) server_negotiation.cc:299] Negotiation successful
0806 22:47:32.690876 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 22:47:32.691236 5400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:32.686622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:50839 (local address 127.18.45.193:40131)
0806 22:47:32.686679 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:32.686683 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:32.686899 (+ 216us) server_negotiation.cc:407] Connection header received
0806 22:47:32.687078 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:32.687081 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:32.687095 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:32.687134 (+ 39us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:47:32.687496 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:32.688203 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:32.689889 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:32.690483 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:32.690548 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:32.690562 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:32.690852 (+ 290us) server_negotiation.cc:299] Negotiation successful
0806 22:47:32.690905 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 22:47:32.691614 5425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:32.686279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.195:39247 (local address 127.18.45.194:46953)
0806 22:47:32.686822 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:32.686833 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:32.686941 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:32.687215 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:32.687216 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:32.687218 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:47:32.687353 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:32.687356 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:32.688364 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:32.688365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:32.689771 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:32.689774 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:32.690657 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:32.690658 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:32.690732 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:32.690737 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:47:32.691436 (+ 699us) client_negotiation.cc:240] Negotiation successful
0806 22:47:32.691492 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"mutex_wait_us":44,"thread_start_us":195,"threads_started":1}
I20260806 22:47:32.691915 5426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:32.686276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.193:40131 (local address 127.18.45.194:50839)
0806 22:47:32.686790 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:32.686805 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:32.686941 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:32.687224 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:32.687225 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:32.687228 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:47:32.687384 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:32.687387 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:32.688348 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:32.688349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:32.689760 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:32.689764 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:32.690668 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:32.690669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:32.690755 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:32.690759 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:47:32.691574 (+ 815us) client_negotiation.cc:240] Negotiation successful
0806 22:47:32.691628 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"mutex_wait_us":127,"thread_start_us":197,"threads_started":1}
I20260806 22:47:32.692251 5298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b84b761b96784f04826f21bf0828eb5f" candidate_uuid: "e36d326e506840fbab8cfec3b34d8366" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "750754f0df8e44eeacc4b9a6a88794ae" is_pre_election: true
I20260806 22:47:32.692508 5298 raft_consensus.cc:2454] T b84b761b96784f04826f21bf0828eb5f P 750754f0df8e44eeacc4b9a6a88794ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e36d326e506840fbab8cfec3b34d8366 in term 0.
I20260806 22:47:32.692466 4897 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b84b761b96784f04826f21bf0828eb5f" candidate_uuid: "e36d326e506840fbab8cfec3b34d8366" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb8153447c6e45a2ab4d3360bae8c43a" is_pre_election: true
I20260806 22:47:32.692672 4897 raft_consensus.cc:2454] T b84b761b96784f04826f21bf0828eb5f P eb8153447c6e45a2ab4d3360bae8c43a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e36d326e506840fbab8cfec3b34d8366 in term 0.
I20260806 22:47:32.692859 5010 leader_election.cc:302] T b84b761b96784f04826f21bf0828eb5f P e36d326e506840fbab8cfec3b34d8366 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 750754f0df8e44eeacc4b9a6a88794ae, e36d326e506840fbab8cfec3b34d8366; no voters:
I20260806 22:47:32.693032 5419 raft_consensus.cc:2781] T b84b761b96784f04826f21bf0828eb5f P e36d326e506840fbab8cfec3b34d8366 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:47:32.693094 5419 raft_consensus.cc:482] T b84b761b96784f04826f21bf0828eb5f P e36d326e506840fbab8cfec3b34d8366 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:47:32.693122 5419 raft_consensus.cc:3037] T b84b761b96784f04826f21bf0828eb5f P e36d326e506840fbab8cfec3b34d8366 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:47:32.694059 5419 raft_consensus.cc:504] T b84b761b96784f04826f21bf0828eb5f P e36d326e506840fbab8cfec3b34d8366 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "750754f0df8e44eeacc4b9a6a88794ae" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 39247 } } peers { permanent_uuid: "e36d326e506840fbab8cfec3b34d8366" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 39045 } } peers { permanent_uuid: "eb8153447c6e45a2ab4d3360bae8c43a" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 40131 } }
I20260806 22:47:32.694247 5419 leader_election.cc:288] T b84b761b96784f04826f21bf0828eb5f P e36d326e506840fbab8cfec3b34d8366 [CANDIDATE]: Term 1 election: Requested vote from peers 750754f0df8e44eeacc4b9a6a88794ae (127.18.45.195:39247), eb8153447c6e45a2ab4d3360bae8c43a (127.18.45.193:40131)
I20260806 22:47:32.694586 4897 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b84b761b96784f04826f21bf0828eb5f" candidate_uuid: "e36d326e506840fbab8cfec3b34d8366" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb8153447c6e45a2ab4d3360bae8c43a"
I20260806 22:47:32.694684 5298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b84b761b96784f04826f21bf0828eb5f" candidate_uuid: "e36d326e506840fbab8cfec3b34d8366" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "750754f0df8e44eeacc4b9a6a88794ae"
I20260806 22:47:32.694721 4897 raft_consensus.cc:3037] T b84b761b96784f04826f21bf0828eb5f P eb8153447c6e45a2ab4d3360bae8c43a [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:47:32.694779 5298 raft_consensus.cc:3037] T b84b761b96784f04826f21bf0828eb5f P 750754f0df8e44eeacc4b9a6a88794ae [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:47:32.695986 4897 raft_consensus.cc:2454] T b84b761b96784f04826f21bf0828eb5f P eb8153447c6e45a2ab4d3360bae8c43a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e36d326e506840fbab8cfec3b34d8366 in term 1.
I20260806 22:47:32.697249 5012 leader_election.cc:302] T b84b761b96784f04826f21bf0828eb5f P e36d326e506840fbab8cfec3b34d8366 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e36d326e506840fbab8cfec3b34d8366, eb8153447c6e45a2ab4d3360bae8c43a; no voters:
I20260806 22:47:32.697474 5419 raft_consensus.cc:2781] T b84b761b96784f04826f21bf0828eb5f P e36d326e506840fbab8cfec3b34d8366 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:47:32.697690 5419 raft_consensus.cc:686] T b84b761b96784f04826f21bf0828eb5f P e36d326e506840fbab8cfec3b34d8366 [term 1 LEADER]: Becoming Leader. State: Replica: e36d326e506840fbab8cfec3b34d8366, State: Running, Role: LEADER
I20260806 22:47:32.697834 5419 consensus_queue.cc:237] T b84b761b96784f04826f21bf0828eb5f P e36d326e506840fbab8cfec3b34d8366 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "750754f0df8e44eeacc4b9a6a88794ae" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 39247 } } peers { permanent_uuid: "e36d326e506840fbab8cfec3b34d8366" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 39045 } } peers { permanent_uuid: "eb8153447c6e45a2ab4d3360bae8c43a" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 40131 } }
I20260806 22:47:32.698251 5298 raft_consensus.cc:2454] T b84b761b96784f04826f21bf0828eb5f P 750754f0df8e44eeacc4b9a6a88794ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e36d326e506840fbab8cfec3b34d8366 in term 1.
I20260806 22:47:32.699290 4715 catalog_manager.cc:5495] T b84b761b96784f04826f21bf0828eb5f P e36d326e506840fbab8cfec3b34d8366 reported cstate change: term changed from 0 to 1, leader changed from <none> to e36d326e506840fbab8cfec3b34d8366 (127.18.45.194). New cstate: current_term: 1 leader_uuid: "e36d326e506840fbab8cfec3b34d8366" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "750754f0df8e44eeacc4b9a6a88794ae" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 39247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e36d326e506840fbab8cfec3b34d8366" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 39045 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb8153447c6e45a2ab4d3360bae8c43a" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 40131 } health_report { overall_health: UNKNOWN } } }
W20260806 22:47:32.700014 5361 tablet.cc:2369] T b84b761b96784f04826f21bf0828eb5f P 750754f0df8e44eeacc4b9a6a88794ae: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:47:32.700430 4986 tablet.cc:2369] T b84b761b96784f04826f21bf0828eb5f P eb8153447c6e45a2ab4d3360bae8c43a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:47:32.718065 5177 tablet.cc:2369] T b84b761b96784f04826f21bf0828eb5f P e36d326e506840fbab8cfec3b34d8366: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 22:47:32 dist-test-slave-h3ft krb5kdc[3295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056451, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.254@KRBTEST.COM for HTTP/127.18.45.212@KRBTEST.COM
I20260806 22:47:32.752276 5402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:32.748820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54974 (local address 127.18.45.194:39045)
0806 22:47:32.748961 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:32.748963 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:32.748970 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:47:32.749016 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:32.749017 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:32.749039 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:32.749132 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:47:32.749343 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:32.750149 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:32.751201 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:32.751553 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:32.751642 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:32.751862 (+ 220us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:47:32.752027 (+ 165us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:47:32.752123 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:32.752147 (+ 24us) server_negotiation.cc:299] Negotiation successful
0806 22:47:32.752175 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
I20260806 22:47:32.754952 5298 raft_consensus.cc:1261] T b84b761b96784f04826f21bf0828eb5f P 750754f0df8e44eeacc4b9a6a88794ae [term 1 FOLLOWER]: Refusing update from remote peer e36d326e506840fbab8cfec3b34d8366: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:47:32.754946 4897 raft_consensus.cc:1261] T b84b761b96784f04826f21bf0828eb5f P eb8153447c6e45a2ab4d3360bae8c43a [term 1 FOLLOWER]: Refusing update from remote peer e36d326e506840fbab8cfec3b34d8366: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:47:32.755422 5427 consensus_queue.cc:1035] T b84b761b96784f04826f21bf0828eb5f P e36d326e506840fbab8cfec3b34d8366 [LEADER]: Connected to new peer: Peer: permanent_uuid: "750754f0df8e44eeacc4b9a6a88794ae" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 39247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:47:32.755693 5419 consensus_queue.cc:1035] T b84b761b96784f04826f21bf0828eb5f P e36d326e506840fbab8cfec3b34d8366 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb8153447c6e45a2ab4d3360bae8c43a" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 40131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:47:32.769274 5437 mvcc.cc:205] Tried to move back new op lower bound from 7315687230480195584 to 7315687230251311104. Current Snapshot: MvccSnapshot[applied={T|T < 7315687230480195584}]
I20260806 22:47:32.769639 5438 mvcc.cc:205] Tried to move back new op lower bound from 7315687230480195584 to 7315687230251311104. Current Snapshot: MvccSnapshot[applied={T|T < 7315687230480195584}]
I20260806 22:47:32.777464 5435 mvcc.cc:205] Tried to move back new op lower bound from 7315687230480195584 to 7315687230251311104. Current Snapshot: MvccSnapshot[applied={T|T < 7315687230480195584 or (T in {7315687230480195584})}]
I20260806 22:47:32.778539 4715 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60370:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:47:32.778689 4715 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60370:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:47:32.780007 4715 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0c72db8fbc444b13bf1f88a9a4762d66: Sending DeleteTablet for 3 replicas of tablet b84b761b96784f04826f21bf0828eb5f
I20260806 22:47:32.780526 5272 tablet_service.cc:1514] Processing DeleteTablet for tablet b84b761b96784f04826f21bf0828eb5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:47:32 UTC) from {username='kudu', principal='kudu/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:41322
I20260806 22:47:32.781180 5077 tablet_service.cc:1514] Processing DeleteTablet for tablet b84b761b96784f04826f21bf0828eb5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:47:32 UTC) from {username='kudu', principal='kudu/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:54966
I20260806 22:47:32.781459 4889 tablet_service.cc:1514] Processing DeleteTablet for tablet b84b761b96784f04826f21bf0828eb5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:47:32 UTC) from {username='kudu', principal='kudu/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:59830
I20260806 22:47:32.782606 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 4768
I20260806 22:47:32.799186 5450 tablet_replica.cc:330] T b84b761b96784f04826f21bf0828eb5f P e36d326e506840fbab8cfec3b34d8366: stopping tablet replica
I20260806 22:47:32.806766 5450 raft_consensus.cc:2229] T b84b761b96784f04826f21bf0828eb5f P e36d326e506840fbab8cfec3b34d8366 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:47:32.807173 5450 raft_consensus.cc:2258] T b84b761b96784f04826f21bf0828eb5f P e36d326e506840fbab8cfec3b34d8366 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:47:32.807557 5449 tablet_replica.cc:330] T b84b761b96784f04826f21bf0828eb5f P 750754f0df8e44eeacc4b9a6a88794ae: stopping tablet replica
I20260806 22:47:32.808240 5450 ts_tablet_manager.cc:1905] T b84b761b96784f04826f21bf0828eb5f P e36d326e506840fbab8cfec3b34d8366: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:47:32.810425 5450 ts_tablet_manager.cc:1918] T b84b761b96784f04826f21bf0828eb5f P e36d326e506840fbab8cfec3b34d8366: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:47:32.810707 5449 raft_consensus.cc:2229] T b84b761b96784f04826f21bf0828eb5f P 750754f0df8e44eeacc4b9a6a88794ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:47:32.810948 5449 raft_consensus.cc:2258] T b84b761b96784f04826f21bf0828eb5f P 750754f0df8e44eeacc4b9a6a88794ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:47:32.811223 5450 log.cc:1196] T b84b761b96784f04826f21bf0828eb5f P e36d326e506840fbab8cfec3b34d8366: Deleting WAL directory at /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-1/wal/wals/b84b761b96784f04826f21bf0828eb5f
I20260806 22:47:32.811770 5450 ts_tablet_manager.cc:1939] T b84b761b96784f04826f21bf0828eb5f P e36d326e506840fbab8cfec3b34d8366: Deleting consensus metadata
I20260806 22:47:32.812480 4700 catalog_manager.cc:4841] TS e36d326e506840fbab8cfec3b34d8366 (127.18.45.194:39045): tablet b84b761b96784f04826f21bf0828eb5f (table test-table [id=1ec22b6c5e9842b781911044cbf13cf1]) successfully deleted
W20260806 22:47:32.813043 4700 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.45.193:40131 (error 108)
I20260806 22:47:32.814306 5398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:32.814155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.193:40131 (local address 127.0.0.1:59836)
0806 22:47:32.814219 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:32.814260 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.193:40131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260806 22:47:32.814586 4700 catalog_manager.cc:4584] TS eb8153447c6e45a2ab4d3360bae8c43a (127.18.45.193:40131): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b84b761b96784f04826f21bf0828eb5f: Network error: Client connection negotiation failed: client connection to 127.18.45.193:40131: connect: Connection refused (error 111)
I20260806 22:47:32.814684 5449 ts_tablet_manager.cc:1905] T b84b761b96784f04826f21bf0828eb5f P 750754f0df8e44eeacc4b9a6a88794ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:47:32.814906 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 4989
I20260806 22:47:32.820693 5449 ts_tablet_manager.cc:1918] T b84b761b96784f04826f21bf0828eb5f P 750754f0df8e44eeacc4b9a6a88794ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:47:32.820816 5449 log.cc:1196] T b84b761b96784f04826f21bf0828eb5f P 750754f0df8e44eeacc4b9a6a88794ae: Deleting WAL directory at /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786056275803658-18615-0/minicluster-data/ts-2/wal/wals/b84b761b96784f04826f21bf0828eb5f
I20260806 22:47:32.821131 5449 ts_tablet_manager.cc:1939] T b84b761b96784f04826f21bf0828eb5f P 750754f0df8e44eeacc4b9a6a88794ae: Deleting consensus metadata
I20260806 22:47:32.821792 4698 catalog_manager.cc:4841] TS 750754f0df8e44eeacc4b9a6a88794ae (127.18.45.195:39247): tablet b84b761b96784f04826f21bf0828eb5f (table test-table [id=1ec22b6c5e9842b781911044cbf13cf1]) successfully deleted
I20260806 22:47:32.830888 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 5179
I20260806 22:47:32.847947 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 4674
I20260806 22:47:32.864462 18615 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 22:47:33.222295 18615 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 22:47:33.222518 18615 mini_ranger.cc:66] Stopping Ranger...
I20260806 22:47:33.578469 18615 mini_ranger.cc:68] Stopped Ranger
2026-08-06 22:47:33.578 UTC [3335] LOG: received smart shutdown request
2026-08-06 22:47:33.583 UTC [3335] LOG: background worker "logical replication launcher" (PID 3349) exited with exit code 1
2026-08-06 22:47:33.583 UTC [3341] LOG: shutting down
2026-08-06 22:47:33.751 UTC [3335] LOG: database system is shut down
2026-08-06T22:47:33Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (57025 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:47:33 dist-test-slave-h3ft krb5kdc[5469](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:47:33 dist-test-slave-h3ft krb5kdc[5469](info): set up 2 sockets
Aug 06 22:47:33 dist-test-slave-h3ft krb5kdc[5469](info): commencing operation
krb5kdc: starting...
W20260806 22:47:35.157928 18615 mini_kdc.cc:121] Time spent starting KDC: real 1.247s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Aug 06 22:47:35 dist-test-slave-h3ft krb5kdc[5469](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056455, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:47:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:47:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.45.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab.
Entry for principal kudu/127.18.45.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab.
I20260806 22:47:35.307256 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:39795
--webserver_interface=127.18.45.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:34697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.45.254:39795,127.18.45.253:45037,127.18.45.252:35489
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:47:35.405536 5487 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:35.405802 5487 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:35.405830 5487 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:35.407147 5487 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:47:35.407207 5487 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:35.407222 5487 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:35.407235 5487 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:47:35.407248 5487 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:47:35.408958 5487 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:34697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.45.254:39795,127.18.45.253:45037,127.18.45.252:35489
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:39795
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.45.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5487
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:47:35.409291 5487 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:35.409595 5487 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:35.415441 5502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:35.415469 5500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:35.415478 5506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:35.416195 5487 server_base.cc:1034] running on GCE node
I20260806 22:47:35.416463 5487 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:35.416774 5487 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:35.417955 5487 hybrid_clock.cc:648] HybridClock initialized: now 1786056455417931 us; error 44 us; skew 500 ppm
Aug 06 22:47:35 dist-test-slave-h3ft krb5kdc[5469](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056455, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:35.421041 5487 init.cc:377] Logged in from keytab as kudu/127.18.45.254@KRBTEST.COM (short username kudu)
I20260806 22:47:35.421599 5487 webserver.cc:460] Webserver started at http://127.18.45.254:35213/ using document root <none> and password file <none>
I20260806 22:47:35.421903 5487 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:35.421957 5487 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:35.422116 5487 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:47:35.423497 5487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/minicluster-data/master-0/data/instance:
uuid: "f85c90dc1cc14b5fa7a2c6635823fde2"
format_stamp: "Formatted at 2026-08-06 22:47:35 on dist-test-slave-h3ft"
server_key: "e77e03fa065377032e2ae520abeb0076"
server_key_iv: "e87e15f4aa4bfadb14bf42e74f6cc51a"
server_key_version: "encryptionkey@0"
I20260806 22:47:35.423925 5487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/minicluster-data/master-0/wal/instance:
uuid: "f85c90dc1cc14b5fa7a2c6635823fde2"
format_stamp: "Formatted at 2026-08-06 22:47:35 on dist-test-slave-h3ft"
server_key: "e77e03fa065377032e2ae520abeb0076"
server_key_iv: "e87e15f4aa4bfadb14bf42e74f6cc51a"
server_key_version: "encryptionkey@0"
I20260806 22:47:35.425495 5487 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:47:35.426465 5515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:35.426659 5487 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:47:35.426757 5487 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/minicluster-data/master-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/minicluster-data/master-0/wal
uuid: "f85c90dc1cc14b5fa7a2c6635823fde2"
format_stamp: "Formatted at 2026-08-06 22:47:35 on dist-test-slave-h3ft"
server_key: "e77e03fa065377032e2ae520abeb0076"
server_key_iv: "e87e15f4aa4bfadb14bf42e74f6cc51a"
server_key_version: "encryptionkey@0"
I20260806 22:47:35.426837 5487 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:35.458827 5487 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:35.461732 5487 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:35.461990 5487 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:35.467985 5487 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.254:39795
I20260806 22:47:35.468029 5587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.254:39795 every 8 connection(s)
I20260806 22:47:35.468519 5487 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
I20260806 22:47:35.469714 5588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:47:35.471834 5588 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39795 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:47:35.475297 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 5487
I20260806 22:47:35.475457 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/minicluster-data/master-0/wal/instance
I20260806 22:47:35.475700 18615 external_mini_cluster.cc:1428] Setting key cd5429d02c795d290400cf0a81c12a5c
Aug 06 22:47:35 dist-test-slave-h3ft krb5kdc[5469](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056455, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.254@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:47:35.484179 5590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:35.472772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39795 (local address 127.0.0.1:49290)
0806 22:47:35.473039 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:35.473079 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:35.473616 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:35.473922 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:35.473924 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:35.474242 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:35.474439 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:35.474443 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:35.475738 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:35.475740 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:35.476945 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:35.476949 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:35.477501 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:35.477503 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:35.477596 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:35.478282 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:35.478298 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:35.480269 (+ 1971us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:35.482326 (+ 2057us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:35.482333 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:35.482336 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:35.482560 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:35.482849 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:35.482850 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:35.482850 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:35.482945 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:35.483441 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:35.483447 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:35.483519 (+ 72us) client_negotiation.cc:769] Sending connection context
0806 22:47:35.483638 (+ 119us) client_negotiation.cc:240] Negotiation successful
0806 22:47:35.483929 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"mutex_wait_us":120,"thread_start_us":118,"threads_started":1}
I20260806 22:47:35.484182 5591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:35.472806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49290 (local address 127.18.45.254:39795)
0806 22:47:35.473051 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:35.473063 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:35.473186 (+ 123us) server_negotiation.cc:407] Connection header received
0806 22:47:35.473765 (+ 579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:35.473767 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:35.473787 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:35.473879 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:35.474535 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:35.475640 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:35.477047 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:35.477401 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:35.477508 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:35.480383 (+ 2875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:35.480398 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:35.480399 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:35.480406 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:35.482213 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:35.482665 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:35.482671 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:35.482671 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:35.482767 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:35.483093 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:35.483094 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:35.483095 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:35.483363 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:35.483469 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:35.483684 (+ 215us) server_negotiation.cc:299] Negotiation successful
0806 22:47:35.483877 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":183,"thread_start_us":102,"threads_started":1}
I20260806 22:47:35.485150 5588 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.45.253" port: 45037 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:47:35.485633 5590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:35.485487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.253:45037 (local address 127.0.0.1:46814)
0806 22:47:35.485547 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:35.485604 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.253:45037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260806 22:47:35.485844 5517 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.45.253:45037: connect: Connection refused (error 111)
I20260806 22:47:35.486435 5590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:35.486287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.253:45037 (local address 127.0.0.1:46818)
0806 22:47:35.486382 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:35.486413 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.253:45037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
W20260806 22:47:35.486615 5588 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.45.253:45037: Network error: Client connection negotiation failed: client connection to 127.18.45.253:45037: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.18.45.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.253.keytab.
Entry for principal kudu/127.18.45.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.253.keytab.
I20260806 22:47:35.510507 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.253:45037
--webserver_interface=127.18.45.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.253.keytab
--principal=kudu/127.18.45.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:34697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.45.254:39795,127.18.45.253:45037,127.18.45.252:35489
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:47:35.535874 5588 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.45.253" port: 45037 } attempt: 1
I20260806 22:47:35.536542 5590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:35.536375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.253:45037 (local address 127.0.0.1:46820)
0806 22:47:35.536452 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:35.536503 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.253:45037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260806 22:47:35.536954 5590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:35.536863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.253:45037 (local address 127.0.0.1:46834)
0806 22:47:35.536907 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:35.536934 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.253:45037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260806 22:47:35.537106 5588 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.45.253:45037: Network error: Client connection negotiation failed: client connection to 127.18.45.253:45037: connect: Connection refused (error 111)
I20260806 22:47:35.605286 5588 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.45.253" port: 45037 } attempt: 2
I20260806 22:47:35.606011 5590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:35.605791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.253:45037 (local address 127.0.0.1:46838)
0806 22:47:35.605922 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:35.605966 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.253:45037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260806 22:47:35.606544 5590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:35.606419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.253:45037 (local address 127.0.0.1:46840)
0806 22:47:35.606472 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:35.606509 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.253:45037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260806 22:47:35.606673 5588 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.45.253:45037: Network error: Client connection negotiation failed: client connection to 127.18.45.253:45037: connect: Connection refused (error 111)
W20260806 22:47:35.606513 5600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:35.606876 5600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:35.606936 5600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:35.608273 5600 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:47:35.608346 5600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:35.608373 5600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:35.608414 5600 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:47:35.608443 5600 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:47:35.610190 5600 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:34697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.18.45.254:39795,127.18.45.253:45037,127.18.45.252:35489
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.253.keytab
--principal=kudu/127.18.45.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.253:45037
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.45.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5600
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:47:35.610544 5600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:35.610828 5600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:35.617067 5605 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:35.617025 5606 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:35.617033 5608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:35.617285 5600 server_base.cc:1034] running on GCE node
I20260806 22:47:35.617725 5600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:35.618044 5600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:35.619234 5600 hybrid_clock.cc:648] HybridClock initialized: now 1786056455619194 us; error 63 us; skew 500 ppm
Aug 06 22:47:35 dist-test-slave-h3ft krb5kdc[5469](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056455, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:35.622745 5600 init.cc:377] Logged in from keytab as kudu/127.18.45.253@KRBTEST.COM (short username kudu)
I20260806 22:47:35.623399 5600 webserver.cc:460] Webserver started at http://127.18.45.253:43075/ using document root <none> and password file <none>
I20260806 22:47:35.623692 5600 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:35.623764 5600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:35.623935 5600 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:47:35.625203 5600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/minicluster-data/master-1/data/instance:
uuid: "4ebab2fd38154221bf1c868b99242279"
format_stamp: "Formatted at 2026-08-06 22:47:35 on dist-test-slave-h3ft"
server_key: "800c1aa9a39c826cef9106ccb26fc9a2"
server_key_iv: "c6605c116724c8363d51c0d17c6a8588"
server_key_version: "encryptionkey@0"
I20260806 22:47:35.625634 5600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/minicluster-data/master-1/wal/instance:
uuid: "4ebab2fd38154221bf1c868b99242279"
format_stamp: "Formatted at 2026-08-06 22:47:35 on dist-test-slave-h3ft"
server_key: "800c1aa9a39c826cef9106ccb26fc9a2"
server_key_iv: "c6605c116724c8363d51c0d17c6a8588"
server_key_version: "encryptionkey@0"
I20260806 22:47:35.627270 5600 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:47:35.628208 5618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:35.628405 5600 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:47:35.628525 5600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/minicluster-data/master-1/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/minicluster-data/master-1/wal
uuid: "4ebab2fd38154221bf1c868b99242279"
format_stamp: "Formatted at 2026-08-06 22:47:35 on dist-test-slave-h3ft"
server_key: "800c1aa9a39c826cef9106ccb26fc9a2"
server_key_iv: "c6605c116724c8363d51c0d17c6a8588"
server_key_version: "encryptionkey@0"
I20260806 22:47:35.628612 5600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:35.653582 5600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:35.656608 5600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:35.656823 5600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:35.662231 5600 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.253:45037
I20260806 22:47:35.662277 5683 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.253:45037 every 8 connection(s)
I20260806 22:47:35.662782 5600 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/minicluster-data/master-1/data/info.pb
I20260806 22:47:35.663908 5684 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:47:35.665863 5684 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39795 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:47:35.668165 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 5600
I20260806 22:47:35.668290 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/minicluster-data/master-1/wal/instance
I20260806 22:47:35.668515 18615 external_mini_cluster.cc:1428] Setting key aa26308389b6a846c5bb2ce69845e388
Aug 06 22:47:35 dist-test-slave-h3ft krb5kdc[5469](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056455, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.253@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:47:35.677939 5591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:35.666785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49300 (local address 127.18.45.254:39795)
0806 22:47:35.666874 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:35.666876 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:35.667171 (+ 295us) server_negotiation.cc:407] Connection header received
0806 22:47:35.667801 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:35.667803 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:35.667820 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:35.667900 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:35.668931 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:35.669740 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:35.671082 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:35.671325 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:35.671392 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:35.674296 (+ 2904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:35.674299 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:35.674299 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:35.674317 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:35.676040 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:35.676720 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:35.676722 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:35.676722 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:35.676801 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:35.677140 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:35.677141 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:35.677142 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:35.677294 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:35.677373 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:35.677709 (+ 336us) server_negotiation.cc:299] Negotiation successful
0806 22:47:35.677826 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 22:47:35.678189 5687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:35.666751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39795 (local address 127.0.0.1:49300)
0806 22:47:35.667025 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:35.667072 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:35.667670 (+ 598us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:35.668029 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:35.668034 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:35.668467 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:35.668779 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:35.668786 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:35.669897 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:35.669897 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:35.670952 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:35.670956 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:35.671423 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:35.671424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:35.671618 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:35.672327 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:35.672339 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:35.674170 (+ 1831us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:35.676153 (+ 1983us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:35.676162 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:35.676165 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:35.676455 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:35.676884 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:35.676885 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:35.676885 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:35.677055 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:35.677373 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:35.677383 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:35.677556 (+ 173us) client_negotiation.cc:769] Sending connection context
0806 22:47:35.677676 (+ 120us) client_negotiation.cc:240] Negotiation successful
0806 22:47:35.677884 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":125,"threads_started":1}
I20260806 22:47:35.678908 5684 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.45.253" port: 45037 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.18.45.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.252@KRBTEST.COM" created.
Aug 06 22:47:35 dist-test-slave-h3ft krb5kdc[5469](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056455, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.253@KRBTEST.COM for kudu/127.18.45.253@KRBTEST.COM
I20260806 22:47:35.688951 5687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:35.679292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.253:45037 (local address 127.0.0.1:46850)
0806 22:47:35.679346 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:35.679358 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:35.679403 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:35.679784 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:35.679785 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:35.680024 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:35.680100 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:35.680103 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:35.681111 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:35.681113 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:35.681802 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:35.681804 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:35.682357 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:35.682358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:35.682440 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:35.683259 (+ 819us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:35.683286 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:35.684928 (+ 1642us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:35.687533 (+ 2605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:35.687534 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:35.687535 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:35.687813 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:35.688078 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:35.688079 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:35.688080 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:35.688159 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:35.688498 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:35.688514 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:35.688558 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 22:47:35.688651 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 22:47:35.688843 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":84}
I20260806 22:47:35.688968 5691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:35.679391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46850 (local address 127.18.45.253:45037)
0806 22:47:35.679624 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:35.679629 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:35.679648 (+ 19us) server_negotiation.cc:407] Connection header received
0806 22:47:35.679677 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:35.679679 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:35.679691 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:35.679785 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:35.680215 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:35.681006 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:35.681892 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:35.682258 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:35.682358 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:35.685031 (+ 2673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:35.685045 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:35.685046 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:35.685056 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:35.687433 (+ 2377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:35.687914 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:35.687918 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:35.687919 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:35.687981 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:35.688230 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:35.688231 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:35.688232 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:35.688416 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:35.688526 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:35.688688 (+ 162us) server_negotiation.cc:299] Negotiation successful
0806 22:47:35.688791 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":160,"threads_started":1}
I20260806 22:47:35.689783 5684 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.45.252" port: 35489 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:47:35.690259 5687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:35.690117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.252:35489 (local address 127.0.0.1:45580)
0806 22:47:35.690172 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:35.690232 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.252:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260806 22:47:35.690464 5621 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.45.252:35489: connect: Connection refused (error 111)
I20260806 22:47:35.691160 5687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:35.690952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.252:35489 (local address 127.0.0.1:45594)
0806 22:47:35.691088 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:35.691126 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.252:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
W20260806 22:47:35.691313 5684 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.45.252:35489: Network error: Client connection negotiation failed: client connection to 127.18.45.252:35489: connect: Connection refused (error 111)
I20260806 22:47:35.697924 5588 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.45.253" port: 45037 } attempt: 3
Aug 06 22:47:35 dist-test-slave-h3ft krb5kdc[5469](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056455, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.254@KRBTEST.COM for kudu/127.18.45.253@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.252.keytab.
Entry for principal kudu/127.18.45.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.252.keytab.
I20260806 22:47:35.706583 5691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:35.698648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46852 (local address 127.18.45.253:45037)
0806 22:47:35.698724 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:35.698727 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:35.698741 (+ 14us) server_negotiation.cc:407] Connection header received
0806 22:47:35.698822 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:35.698823 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:35.698846 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:35.698990 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:35.699455 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:35.700235 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:35.700980 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:35.701237 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:35.701334 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:35.703631 (+ 2297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:35.703633 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:35.703633 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:35.703644 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:35.705255 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:35.705648 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:35.705652 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:35.705652 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:35.705717 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:35.705953 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:35.705954 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:35.705954 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:35.706096 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:35.706178 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:35.706357 (+ 179us) server_negotiation.cc:299] Negotiation successful
0806 22:47:35.706460 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 22:47:35.706583 5590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:35.698520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.253:45037 (local address 127.0.0.1:46852)
0806 22:47:35.698629 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:35.698643 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:35.698739 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:35.699032 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:35.699033 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:35.699241 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:35.699338 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:35.699340 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:35.700332 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:35.700334 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:35.700886 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:35.700887 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:35.701334 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:35.701335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:35.701434 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:35.702026 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:35.702031 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:35.703532 (+ 1501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:35.705355 (+ 1823us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:35.705357 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:35.705357 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:35.705560 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:35.705796 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:35.705797 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:35.705797 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:35.705865 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:35.706177 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:35.706179 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:35.706235 (+ 56us) client_negotiation.cc:769] Sending connection context
0806 22:47:35.706317 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 22:47:35.706459 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260806 22:47:35.707198 5588 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.45.252" port: 35489 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:47:35.707613 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.252:35489
--webserver_interface=127.18.45.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.252.keytab
--principal=kudu/127.18.45.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:34697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.45.254:39795,127.18.45.253:45037,127.18.45.252:35489
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:47:35.707676 5590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:35.707538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.252:35489 (local address 127.0.0.1:45610)
0806 22:47:35.707597 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:35.707632 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.252:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260806 22:47:35.708138 5590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:35.707990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.252:35489 (local address 127.0.0.1:45620)
0806 22:47:35.708055 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:35.708099 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.252:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260806 22:47:35.708264 5588 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.45.252:35489: Network error: Client connection negotiation failed: client connection to 127.18.45.252:35489: connect: Connection refused (error 111)
I20260806 22:47:35.739430 5588 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.45.252" port: 35489 } attempt: 1
I20260806 22:47:35.740100 5590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:35.739917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.252:35489 (local address 127.0.0.1:45626)
0806 22:47:35.740002 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:35.740055 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.252:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260806 22:47:35.740571 5590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:35.740436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.252:35489 (local address 127.0.0.1:45634)
0806 22:47:35.740486 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:35.740537 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.252:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 22:47:35.740587 5684 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.45.252" port: 35489 } attempt: 1
W20260806 22:47:35.740775 5588 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.45.252:35489: Network error: Client connection negotiation failed: client connection to 127.18.45.252:35489: connect: Connection refused (error 111)
I20260806 22:47:35.741130 5687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:35.740959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.252:35489 (local address 127.0.0.1:45648)
0806 22:47:35.741049 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:35.741093 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.252:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260806 22:47:35.741559 5687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:35.741446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.252:35489 (local address 127.0.0.1:45656)
0806 22:47:35.741498 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:35.741526 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.252:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260806 22:47:35.741657 5684 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.45.252:35489: Network error: Client connection negotiation failed: client connection to 127.18.45.252:35489: connect: Connection refused (error 111)
W20260806 22:47:35.808741 5694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:35.809072 5694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:35.809115 5694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260806 22:47:35.809803 5684 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.45.252" port: 35489 } attempt: 2
I20260806 22:47:35.810461 5687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:35.810302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.252:35489 (local address 127.0.0.1:45662)
0806 22:47:35.810376 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:35.810422 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.252:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260806 22:47:35.810479 5694 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:47:35.810546 5694 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:35.810572 5694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:35.810611 5694 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:47:35.810627 5694 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:47:35.810910 5687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:35.810745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.252:35489 (local address 127.0.0.1:45672)
0806 22:47:35.810821 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:35.810869 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.252:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260806 22:47:35.811065 5684 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.45.252:35489: Network error: Client connection negotiation failed: client connection to 127.18.45.252:35489: connect: Connection refused (error 111)
I20260806 22:47:35.812849 5694 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:34697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.18.45.254:39795,127.18.45.253:45037,127.18.45.252:35489
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.252.keytab
--principal=kudu/127.18.45.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.252:35489
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.45.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5694
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:47:35.813202 5694 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:35.813541 5694 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 22:47:35.815927 5588 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.45.252" port: 35489 } attempt: 2
I20260806 22:47:35.816416 5590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:35.816271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.252:35489 (local address 127.0.0.1:45688)
0806 22:47:35.816337 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:35.816382 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.252:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260806 22:47:35.816869 5590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:35.816724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.252:35489 (local address 127.0.0.1:45700)
0806 22:47:35.816788 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:35.816831 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.252:35489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260806 22:47:35.816982 5588 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.45.252:35489: Network error: Client connection negotiation failed: client connection to 127.18.45.252:35489: connect: Connection refused (error 111)
W20260806 22:47:35.819397 5702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:35.819429 5701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:35.819540 5704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:35.819885 5694 server_base.cc:1034] running on GCE node
I20260806 22:47:35.820254 5694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:35.820626 5694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:35.821810 5694 hybrid_clock.cc:648] HybridClock initialized: now 1786056455821740 us; error 80 us; skew 500 ppm
Aug 06 22:47:35 dist-test-slave-h3ft krb5kdc[5469](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056455, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:35.825663 5694 init.cc:377] Logged in from keytab as kudu/127.18.45.252@KRBTEST.COM (short username kudu)
I20260806 22:47:35.826294 5694 webserver.cc:460] Webserver started at http://127.18.45.252:34729/ using document root <none> and password file <none>
I20260806 22:47:35.826610 5694 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:35.826695 5694 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:35.826886 5694 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:47:35.828349 5694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/minicluster-data/master-2/data/instance:
uuid: "843989e8da154295a4251b0b8add6d0d"
format_stamp: "Formatted at 2026-08-06 22:47:35 on dist-test-slave-h3ft"
server_key: "4ccd8c5093e650bc6be8182ca7e7f545"
server_key_iv: "05bbc57b12042c79ffcc97884e7a8d4d"
server_key_version: "encryptionkey@0"
I20260806 22:47:35.828836 5694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/minicluster-data/master-2/wal/instance:
uuid: "843989e8da154295a4251b0b8add6d0d"
format_stamp: "Formatted at 2026-08-06 22:47:35 on dist-test-slave-h3ft"
server_key: "4ccd8c5093e650bc6be8182ca7e7f545"
server_key_iv: "05bbc57b12042c79ffcc97884e7a8d4d"
server_key_version: "encryptionkey@0"
I20260806 22:47:35.830865 5694 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260806 22:47:35.832026 5714 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:35.832401 5694 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:47:35.832540 5694 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/minicluster-data/master-2/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/minicluster-data/master-2/wal
uuid: "843989e8da154295a4251b0b8add6d0d"
format_stamp: "Formatted at 2026-08-06 22:47:35 on dist-test-slave-h3ft"
server_key: "4ccd8c5093e650bc6be8182ca7e7f545"
server_key_iv: "05bbc57b12042c79ffcc97884e7a8d4d"
server_key_version: "encryptionkey@0"
I20260806 22:47:35.832659 5694 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:35.849519 5694 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:35.852209 5694 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:35.852389 5694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:35.857548 5694 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.252:35489
I20260806 22:47:35.857622 5791 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.252:35489 every 8 connection(s)
I20260806 22:47:35.858062 5694 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/minicluster-data/master-2/data/info.pb
I20260806 22:47:35.859254 5792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:47:35.861202 5792 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39795 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:47:35.866318 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 5694
I20260806 22:47:35.866494 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786056275803658-18615-0/minicluster-data/master-2/wal/instance
I20260806 22:47:35.866732 18615 external_mini_cluster.cc:1428] Setting key 66e7a67ab9cc7a9641c232068dcddf6f
Aug 06 22:47:35 dist-test-slave-h3ft krb5kdc[5469](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056455, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.252@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:47:35.873292 5794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:35.862166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39795 (local address 127.0.0.1:49302)
0806 22:47:35.862485 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:35.862535 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:35.863197 (+ 662us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:35.863545 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:35.863549 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:35.863881 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:35.864171 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:35.864176 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:35.865018 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:35.865020 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:35.865985 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:35.865987 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:35.866525 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:35.866526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:35.866694 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:35.867498 (+ 804us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:35.867516 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:35.869280 (+ 1764us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:35.871415 (+ 2135us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:35.871424 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:35.871426 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:35.871822 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:35.872058 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:35.872059 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:35.872059 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:35.872232 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:35.872538 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:35.872546 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:35.872794 (+ 248us) client_negotiation.cc:769] Sending connection context
0806 22:47:35.872909 (+ 115us) client_negotiation.cc:240] Negotiation successful
0806 22:47:35.873060 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":128,"threads_started":1}
I20260806 22:47:35.873286 5591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:35.862213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49302 (local address 127.18.45.254:39795)
0806 22:47:35.862429 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:35.862432 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:35.862665 (+ 233us) server_negotiation.cc:407] Connection header received
0806 22:47:35.863339 (+ 674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:35.863341 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:35.863357 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:35.863436 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:35.864311 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:35.864899 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:35.866171 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:35.866388 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:35.866515 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:35.869378 (+ 2863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:35.869380 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:35.869380 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:35.869390 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:35.871313 (+ 1923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:35.871913 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:35.871915 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:35.871915 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:35.871978 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:35.872316 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:35.872317 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:35.872318 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:35.872461 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:35.872520 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:35.873066 (+ 546us) server_negotiation.cc:299] Negotiation successful
0806 22:47:35.873166 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167}
Aug 06 22:47:35 dist-test-slave-h3ft krb5kdc[5469](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056455, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.45.252@KRBTEST.COM
I20260806 22:47:35.874039 5792 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.45.253" port: 45037 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:47:35.877946 5796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:35.867284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45704 (local address 127.18.45.252:35489)
0806 22:47:35.867561 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:35.867567 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:35.867591 (+ 24us) server_negotiation.cc:407] Connection header received
0806 22:47:35.867657 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:35.867659 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:35.867684 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:35.867793 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:35.868593 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:35.869489 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:35.870292 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:35.870793 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:35.870880 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:35.874255 (+ 3375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:35.874264 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:35.874264 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:35.874273 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:35.876451 (+ 2178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:35.876993 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:35.876996 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:35.876997 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:35.877085 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:35.877314 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:35.877315 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:35.877316 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:35.877528 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:35.877709 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:35.877759 (+ 50us) server_negotiation.cc:299] Negotiation successful
0806 22:47:35.877836 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":132,"threads_started":1}
Aug 06 22:47:35 dist-test-slave-h3ft krb5kdc[5469](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056455, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.252@KRBTEST.COM for kudu/127.18.45.253@KRBTEST.COM
I20260806 22:47:35.881706 5794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:35.874387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.253:45037 (local address 127.0.0.1:46866)
0806 22:47:35.874452 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:35.874465 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:35.874511 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:35.874659 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:35.874660 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:35.874821 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:35.874907 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:35.874908 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:35.875798 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:35.875800 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:35.876621 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:35.876623 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:35.877110 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:35.877111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:35.877206 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:35.877586 (+ 380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:35.877594 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:35.879141 (+ 1547us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:35.880695 (+ 1554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:35.880696 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:35.880696 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:35.880898 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:35.881112 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:35.881112 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:35.881112 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:35.881163 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:35.881436 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:35.881436 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:35.881472 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 22:47:35.881522 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 22:47:35.881595 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260806 22:47:35.881724 5691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:35.874470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46866 (local address 127.18.45.253:45037)
0806 22:47:35.874526 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:35.874529 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:35.874556 (+ 27us) server_negotiation.cc:407] Connection header received
0806 22:47:35.874584 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:35.874585 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:35.874595 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:35.874636 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:35.875039 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:35.875702 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:35.876710 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:35.877026 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:35.877086 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:35.879218 (+ 2132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:35.879219 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:35.879220 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:35.879226 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:35.880615 (+ 1389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:35.880966 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:35.880967 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:35.880968 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:35.881039 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:35.881226 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:35.881227 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:35.881228 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:35.881373 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:35.881442 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:35.881546 (+ 104us) server_negotiation.cc:299] Negotiation successful
0806 22:47:35.881634 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 22:47:35.882187 5792 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.45.252" port: 35489 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:47:35 dist-test-slave-h3ft krb5kdc[5469](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056455, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
Aug 06 22:47:35 dist-test-slave-h3ft krb5kdc[5469](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056455, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.252@KRBTEST.COM for kudu/127.18.45.252@KRBTEST.COM
I20260806 22:47:35.887566 5591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:35.878994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49312 (local address 127.18.45.254:39795)
0806 22:47:35.879060 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:35.879062 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:35.879132 (+ 70us) server_negotiation.cc:407] Connection header received
0806 22:47:35.879470 (+ 338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:35.879471 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:35.879482 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:35.879555 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:35.880202 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:35.880701 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:35.881675 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:35.881957 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:35.882029 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:35.884601 (+ 2572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:35.884603 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:35.884603 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:35.884610 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:35.886179 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:35.886595 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:35.886596 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:35.886596 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:35.886653 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:35.886871 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:35.886872 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:35.886873 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:35.887074 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:35.887318 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:35.887362 (+ 44us) server_negotiation.cc:299] Negotiation successful
0806 22:47:35.887442 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 22:47:35.889377 5794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:35.882506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.252:35489 (local address 127.0.0.1:45710)
0806 22:47:35.882546 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:35.882557 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:35.882595 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:35.882776 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:35.882777 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:35.882893 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:35.882924 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:35.882925 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:35.883849 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:35.883850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:35.884454 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:35.884457 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:35.884857 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:35.884858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:35.884913 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:35.885262 (+ 349us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:35.885266 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:35.886429 (+ 1163us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:35.888307 (+ 1878us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:35.888308 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:35.888309 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:35.888544 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:35.888750 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:35.888751 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:35.888751 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:35.888825 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:35.889062 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:35.889063 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:35.889112 (+ 49us) client_negotiation.cc:769] Sending connection context
0806 22:47:35.889200 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 22:47:35.889311 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260806 22:47:35.889567 5796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:35.882590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45710 (local address 127.18.45.252:35489)
0806 22:47:35.882654 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:35.882656 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:35.882666 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:47:35.882687 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:35.882688 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:35.882701 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:35.882768 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:35.883065 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:35.883771 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:35.884541 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:35.884782 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:35.884860 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:35.886570 (+ 1710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:35.886571 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:35.886572 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:35.886580 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:35.888237 (+ 1657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:35.888617 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:35.888618 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:35.888619 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:35.888684 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:35.888894 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:35.888895 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:35.888895 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:35.889000 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:35.889047 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:35.889240 (+ 193us) server_negotiation.cc:299] Negotiation successful
0806 22:47:35.889367 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":39}
I20260806 22:47:35.891798 5792 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 843989e8da154295a4251b0b8add6d0d: Bootstrap starting.
I20260806 22:47:35.893185 5792 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 843989e8da154295a4251b0b8add6d0d: Neither blocks nor log segments found. Creating new log.
Aug 06 22:47:35 dist-test-slave-h3ft krb5kdc[5469](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056455, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.45.253@KRBTEST.COM
I20260806 22:47:35.893589 5792 log.cc:824] T 00000000000000000000000000000000 P 843989e8da154295a4251b0b8add6d0d: Log is configured to *not* fsync() on all Append() calls
I20260806 22:47:35.894682 5792 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 843989e8da154295a4251b0b8add6d0d: No bootstrap required, opened a new log
I20260806 22:47:35.897188 5691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:35.888486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46872 (local address 127.18.45.253:45037)
0806 22:47:35.888542 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:35.888544 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:35.888552 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:47:35.888576 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:35.888576 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:35.888588 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:35.888645 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:35.889258 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:35.889971 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:35.890834 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:35.891118 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:35.891178 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:35.893934 (+ 2756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:35.893936 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:35.893936 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:35.893944 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:35.895827 (+ 1883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:35.896226 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:35.896228 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:35.896228 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:35.896295 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:35.896551 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:35.896553 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:35.896553 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:35.896711 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:35.896947 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:35.896991 (+ 44us) server_negotiation.cc:299] Negotiation successful
0806 22:47:35.897084 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 22:47:35.897606 5792 raft_consensus.cc:348] T 00000000000000000000000000000000 P 843989e8da154295a4251b0b8add6d0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f85c90dc1cc14b5fa7a2c6635823fde2" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39795 } } peers { permanent_uuid: "4ebab2fd38154221bf1c868b99242279" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 45037 } } peers { permanent_uuid: "843989e8da154295a4251b0b8add6d0d" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 35489 } }
I20260806 22:47:35.897815 5792 raft_consensus.cc:374] T 00000000000000000000000000000000 P 843989e8da154295a4251b0b8add6d0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:47:35.897883 5792 raft_consensus.cc:729] T 00000000000000000000000000000000 P 843989e8da154295a4251b0b8add6d0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 843989e8da154295a4251b0b8add6d0d, State: Initialized, Role: FOLLOWER
I20260806 22:47:35.898039 5792 consensus_queue.cc:260] T 00000000000000000000000000000000 P 843989e8da154295a4251b0b8add6d0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f85c90dc1cc14b5fa7a2c6635823fde2" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39795 } } peers { permanent_uuid: "4ebab2fd38154221bf1c868b99242279" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 45037 } } peers { permanent_uuid: "843989e8da154295a4251b0b8add6d0d" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 35489 } }
I20260806 22:47:35.898376 5798 sys_catalog.cc:455] T 00000000000000000000000000000000 P 843989e8da154295a4251b0b8add6d0d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f85c90dc1cc14b5fa7a2c6635823fde2" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39795 } } peers { permanent_uuid: "4ebab2fd38154221bf1c868b99242279" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 45037 } } peers { permanent_uuid: "843989e8da154295a4251b0b8add6d0d" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 35489 } } }
I20260806 22:47:35.898470 5792 sys_catalog.cc:564] T 00000000000000000000000000000000 P 843989e8da154295a4251b0b8add6d0d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:47:35.898485 5798 sys_catalog.cc:458] T 00000000000000000000000000000000 P 843989e8da154295a4251b0b8add6d0d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:47:35.899159 18615 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260806 22:47:35.900837 5814 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 843989e8da154295a4251b0b8add6d0d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:47:35.900954 5814 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:47:35.902225 5684 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.45.252" port: 35489 } attempt: 3
I20260806 22:47:35.901142 5694 master_runner.cc:186] Error getting master registration for 127.18.45.254:39795: OK, instance_id { permanent_uuid: "f85c90dc1cc14b5fa7a2c6635823fde2" instance_seqno: 1786056455466232 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 22:47:35.904928 5694 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 22:47:35.909607 5591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:35.901369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49318 (local address 127.18.45.254:39795)
0806 22:47:35.901522 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:35.901525 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:35.901534 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:47:35.901562 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:35.901563 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:35.901576 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:35.901639 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:35.902323 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:35.903070 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:35.904320 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:35.904597 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:35.904646 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:35.906374 (+ 1728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:35.906375 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:35.906376 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:35.906384 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:35.908197 (+ 1813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:35.908586 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:35.908588 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:35.908588 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:35.908650 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:35.908870 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:35.908872 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:35.908872 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:35.909056 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:35.909174 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:35.909384 (+ 210us) server_negotiation.cc:299] Negotiation successful
0806 22:47:35.909479 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
Aug 06 22:47:35 dist-test-slave-h3ft krb5kdc[5469](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056455, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.253@KRBTEST.COM for kudu/127.18.45.252@KRBTEST.COM
I20260806 22:47:35.911850 5691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:35.900917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46878 (local address 127.18.45.253:45037)
0806 22:47:35.900968 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:35.900970 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:35.901330 (+ 360us) server_negotiation.cc:407] Connection header received
0806 22:47:35.901394 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:35.901395 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:35.901407 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:35.901462 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:35.902218 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:35.902945 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:35.904394 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:35.904668 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:35.904716 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:35.906630 (+ 1914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:35.906631 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:35.906632 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:35.906909 (+ 277us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:35.909699 (+ 2790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:35.910086 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:35.910088 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:35.910089 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:35.910147 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:35.910371 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:35.910372 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:35.910373 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:35.910532 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:35.911378 (+ 846us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:35.911662 (+ 284us) server_negotiation.cc:299] Negotiation successful
0806 22:47:35.911746 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 22:47:35.913579 5687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:35.903152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.252:35489 (local address 127.0.0.1:45734)
0806 22:47:35.903252 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:35.903261 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:35.903297 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:35.905229 (+ 1932us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:35.905230 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:35.905431 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:35.905483 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:35.905485 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:35.906555 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:35.906556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:35.907369 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:35.907372 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:35.907916 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:35.907918 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:35.907995 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:35.908446 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:35.908451 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:35.910094 (+ 1643us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:35.912315 (+ 2221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:35.912316 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:35.912316 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:35.912540 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:35.912795 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:35.912796 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:35.912796 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:35.912857 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:35.913208 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:35.913208 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:35.913255 (+ 47us) client_negotiation.cc:769] Sending connection context
0806 22:47:35.913366 (+ 111us) client_negotiation.cc:240] Negotiation successful
0806 22:47:35.913472 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83}
I20260806 22:47:35.913820 5826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:35.903364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45734 (local address 127.18.45.252:35489)
0806 22:47:35.905056 (+ 1692us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:35.905058 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:35.905067 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:47:35.905090 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:35.905090 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:35.905101 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:35.905205 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:35.905585 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:35.906460 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:35.907487 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:35.907821 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:35.908140 (+ 319us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:35.910183 (+ 2043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:35.910185 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:35.910185 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:35.910192 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:35.912218 (+ 2026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:35.912648 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:35.912650 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:35.912650 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:35.912723 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:35.912941 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:35.912942 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:35.912942 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:35.913126 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:35.913217 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:35.913397 (+ 180us) server_negotiation.cc:299] Negotiation successful
0806 22:47:35.913476 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1665,"thread_start_us":88,"threads_started":1}
I20260806 22:47:35.915846 5684 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ebab2fd38154221bf1c868b99242279: Bootstrap starting.
I20260806 22:47:35.916324 5588 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.45.252" port: 35489 } attempt: 3
I20260806 22:47:35.916888 5684 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4ebab2fd38154221bf1c868b99242279: Neither blocks nor log segments found. Creating new log.
I20260806 22:47:35.917325 5684 log.cc:824] T 00000000000000000000000000000000 P 4ebab2fd38154221bf1c868b99242279: Log is configured to *not* fsync() on all Append() calls
I20260806 22:47:35.918408 5684 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ebab2fd38154221bf1c868b99242279: No bootstrap required, opened a new log
I20260806 22:47:35.919034 5796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:35.901461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45722 (local address 127.18.45.252:35489)
0806 22:47:35.901701 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:35.901703 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:35.910374 (+ 8671us) server_negotiation.cc:407] Connection header received
0806 22:47:35.910474 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:35.910475 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:35.910487 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:35.910534 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:35.911214 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:35.912209 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:35.913234 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:35.913544 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:35.913588 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:35.915351 (+ 1763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:35.915352 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:35.915353 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:35.915360 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:35.917674 (+ 2314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:35.918098 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:35.918099 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:35.918100 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:35.918162 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:35.918374 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:35.918375 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:35.918376 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:35.918526 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:35.918618 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:35.918831 (+ 213us) server_negotiation.cc:299] Negotiation successful
0806 22:47:35.918915 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216}
I20260806 22:47:35.920826 5684 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4ebab2fd38154221bf1c868b99242279 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f85c90dc1cc14b5fa7a2c6635823fde2" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39795 } } peers { permanent_uuid: "4ebab2fd38154221bf1c868b99242279" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 45037 } } peers { permanent_uuid: "843989e8da154295a4251b0b8add6d0d" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 35489 } }
I20260806 22:47:35.921027 5684 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4ebab2fd38154221bf1c868b99242279 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:47:35.921077 5684 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4ebab2fd38154221bf1c868b99242279 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ebab2fd38154221bf1c868b99242279, State: Initialized, Role: FOLLOWER
I20260806 22:47:35.921226 5684 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4ebab2fd38154221bf1c868b99242279 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f85c90dc1cc14b5fa7a2c6635823fde2" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39795 } } peers { permanent_uuid: "4ebab2fd38154221bf1c868b99242279" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 45037 } } peers { permanent_uuid: "843989e8da154295a4251b0b8add6d0d" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 35489 } }
Aug 06 22:47:35 dist-test-slave-h3ft krb5kdc[5469](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056455, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.254@KRBTEST.COM for kudu/127.18.45.252@KRBTEST.COM
I20260806 22:47:35.921587 5828 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ebab2fd38154221bf1c868b99242279 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f85c90dc1cc14b5fa7a2c6635823fde2" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39795 } } peers { permanent_uuid: "4ebab2fd38154221bf1c868b99242279" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 45037 } } peers { permanent_uuid: "843989e8da154295a4251b0b8add6d0d" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 35489 } } }
I20260806 22:47:35.921646 5684 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4ebab2fd38154221bf1c868b99242279 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:47:35.921687 5828 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ebab2fd38154221bf1c868b99242279 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 22:47:35.924751 5842 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4ebab2fd38154221bf1c868b99242279: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:47:35.924881 5842 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:47:35.925510 5590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:35.916649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.252:35489 (local address 127.0.0.1:45740)
0806 22:47:35.916708 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:35.916720 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:35.916760 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:35.916934 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:35.916936 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:35.917135 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:35.917194 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:35.917197 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:35.918199 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:35.918200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:35.919045 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:35.919048 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:35.919754 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:35.919755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:35.919818 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:35.920275 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:35.920280 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:35.921942 (+ 1662us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:35.924001 (+ 2059us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:35.924002 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:35.924003 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:35.924259 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:35.924516 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:35.924517 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:35.924517 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:35.924599 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:35.925139 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:35.925140 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:35.925193 (+ 53us) client_negotiation.cc:769] Sending connection context
0806 22:47:35.925261 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 22:47:35.925384 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260806 22:47:35.925912 5826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:35.916714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45740 (local address 127.18.45.252:35489)
0806 22:47:35.916790 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:35.916792 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:35.916800 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:47:35.916823 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:35.916824 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:35.916833 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:35.917266 (+ 433us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:35.917299 (+ 33us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:35.918092 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:35.919190 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:35.919661 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:35.920050 (+ 389us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:35.922045 (+ 1995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:35.922047 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:35.922047 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:35.922054 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:35.923924 (+ 1870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:35.924366 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:35.924368 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:35.924369 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:35.924431 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:35.924671 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:35.924672 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:35.924673 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:35.924851 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:35.924919 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:35.925719 (+ 800us) server_negotiation.cc:299] Negotiation successful
0806 22:47:35.925814 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 22:47:35.927886 5588 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f85c90dc1cc14b5fa7a2c6635823fde2: Bootstrap starting.
I20260806 22:47:35.924674 5600 master_runner.cc:186] Error getting master registration for 127.18.45.254:39795: OK, instance_id { permanent_uuid: "f85c90dc1cc14b5fa7a2c6635823fde2" instance_seqno: 1786056455466232 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 22:47:35.928610 5600 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 22:47:35.928875 5588 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f85c90dc1cc14b5fa7a2c6635823fde2: Neither blocks nor log segments found. Creating new log.
I20260806 22:47:35.929327 5588 log.cc:824] T 00000000000000000000000000000000 P f85c90dc1cc14b5fa7a2c6635823fde2: Log is configured to *not* fsync() on all Append() calls
I20260806 22:47:35.930312 5588 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f85c90dc1cc14b5fa7a2c6635823fde2: No bootstrap required, opened a new log
I20260806 22:47:35.932783 5588 raft_consensus.cc:348] T 00000000000000000000000000000000 P f85c90dc1cc14b5fa7a2c6635823fde2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f85c90dc1cc14b5fa7a2c6635823fde2" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39795 } } peers { permanent_uuid: "4ebab2fd38154221bf1c868b99242279" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 45037 } } peers { permanent_uuid: "843989e8da154295a4251b0b8add6d0d" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 35489 } }
I20260806 22:47:35.932962 5588 raft_consensus.cc:374] T 00000000000000000000000000000000 P f85c90dc1cc14b5fa7a2c6635823fde2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:47:35.933033 5588 raft_consensus.cc:729] T 00000000000000000000000000000000 P f85c90dc1cc14b5fa7a2c6635823fde2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f85c90dc1cc14b5fa7a2c6635823fde2, State: Initialized, Role: FOLLOWER
I20260806 22:47:35.933202 5588 consensus_queue.cc:260] T 00000000000000000000000000000000 P f85c90dc1cc14b5fa7a2c6635823fde2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f85c90dc1cc14b5fa7a2c6635823fde2" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39795 } } peers { permanent_uuid: "4ebab2fd38154221bf1c868b99242279" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 45037 } } peers { permanent_uuid: "843989e8da154295a4251b0b8add6d0d" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 35489 } }
I20260806 22:47:35.933539 5845 sys_catalog.cc:455] T 00000000000000000000000000000000 P f85c90dc1cc14b5fa7a2c6635823fde2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f85c90dc1cc14b5fa7a2c6635823fde2" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39795 } } peers { permanent_uuid: "4ebab2fd38154221bf1c868b99242279" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 45037 } } peers { permanent_uuid: "843989e8da154295a4251b0b8add6d0d" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 35489 } } }
I20260806 22:47:35.933622 5588 sys_catalog.cc:564] T 00000000000000000000000000000000 P f85c90dc1cc14b5fa7a2c6635823fde2 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:47:35.933701 5845 sys_catalog.cc:458] T 00000000000000000000000000000000 P f85c90dc1cc14b5fa7a2c6635823fde2 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 22:47:35.936445 5860 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f85c90dc1cc14b5fa7a2c6635823fde2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:47:35.936794 5860 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:47:35.938894 5487 master_runner.cc:263] No leader master found from master f85c90dc1cc14b5fa7a2c6635823fde2
I20260806 22:47:35.938969 5487 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 22:47:35.963243 5522 tablet.cc:2369] T 00000000000000000000000000000000 P f85c90dc1cc14b5fa7a2c6635823fde2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:47:36.016225 5828 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4ebab2fd38154221bf1c868b99242279 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:47:36.016407 5828 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4ebab2fd38154221bf1c868b99242279 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f85c90dc1cc14b5fa7a2c6635823fde2" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39795 } } peers { permanent_uuid: "4ebab2fd38154221bf1c868b99242279" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 45037 } } peers { permanent_uuid: "843989e8da154295a4251b0b8add6d0d" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 35489 } }
I20260806 22:47:36.016804 5828 leader_election.cc:288] T 00000000000000000000000000000000 P 4ebab2fd38154221bf1c868b99242279 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f85c90dc1cc14b5fa7a2c6635823fde2 (127.18.45.254:39795), 843989e8da154295a4251b0b8add6d0d (127.18.45.252:35489)
I20260806 22:47:36.017309 5761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4ebab2fd38154221bf1c868b99242279" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843989e8da154295a4251b0b8add6d0d" is_pre_election: true
I20260806 22:47:36.017519 5761 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 843989e8da154295a4251b0b8add6d0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ebab2fd38154221bf1c868b99242279 in term 0.
I20260806 22:47:36.017834 5621 leader_election.cc:302] T 00000000000000000000000000000000 P 4ebab2fd38154221bf1c868b99242279 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ebab2fd38154221bf1c868b99242279, 843989e8da154295a4251b0b8add6d0d; no voters:
I20260806 22:47:36.018594 5553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4ebab2fd38154221bf1c868b99242279" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f85c90dc1cc14b5fa7a2c6635823fde2" is_pre_election: true
I20260806 22:47:36.018780 5553 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f85c90dc1cc14b5fa7a2c6635823fde2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ebab2fd38154221bf1c868b99242279 in term 0.
I20260806 22:47:36.019250 5828 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4ebab2fd38154221bf1c868b99242279 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:47:36.019330 5828 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4ebab2fd38154221bf1c868b99242279 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:47:36.019356 5828 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4ebab2fd38154221bf1c868b99242279 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:47:36.020532 5828 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4ebab2fd38154221bf1c868b99242279 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f85c90dc1cc14b5fa7a2c6635823fde2" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39795 } } peers { permanent_uuid: "4ebab2fd38154221bf1c868b99242279" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 45037 } } peers { permanent_uuid: "843989e8da154295a4251b0b8add6d0d" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 35489 } }
I20260806 22:47:36.020712 5828 leader_election.cc:288] T 00000000000000000000000000000000 P 4ebab2fd38154221bf1c868b99242279 [CANDIDATE]: Term 1 election: Requested vote from peers f85c90dc1cc14b5fa7a2c6635823fde2 (127.18.45.254:39795), 843989e8da154295a4251b0b8add6d0d (127.18.45.252:35489)
I20260806 22:47:36.021066 5761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4ebab2fd38154221bf1c868b99242279" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "843989e8da154295a4251b0b8add6d0d"
I20260806 22:47:36.021178 5761 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 843989e8da154295a4251b0b8add6d0d [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:47:36.022326 5761 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 843989e8da154295a4251b0b8add6d0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ebab2fd38154221bf1c868b99242279 in term 1.
I20260806 22:47:36.022591 5621 leader_election.cc:302] T 00000000000000000000000000000000 P 4ebab2fd38154221bf1c868b99242279 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ebab2fd38154221bf1c868b99242279, 843989e8da154295a4251b0b8add6d0d; no voters:
I20260806 22:47:36.022945 5553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4ebab2fd38154221bf1c868b99242279" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f85c90dc1cc14b5fa7a2c6635823fde2"
I20260806 22:47:36.023094 5553 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f85c90dc1cc14b5fa7a2c6635823fde2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:47:36.024274 5553 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f85c90dc1cc14b5fa7a2c6635823fde2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ebab2fd38154221bf1c868b99242279 in term 1.
I20260806 22:47:36.025918 5828 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4ebab2fd38154221bf1c868b99242279 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:47:36.026158 5828 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4ebab2fd38154221bf1c868b99242279 [term 1 LEADER]: Becoming Leader. State: Replica: 4ebab2fd38154221bf1c868b99242279, State: Running, Role: LEADER
I20260806 22:47:36.026273 5828 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ebab2fd38154221bf1c868b99242279 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f85c90dc1cc14b5fa7a2c6635823fde2" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39795 } } peers { permanent_uuid: "4ebab2fd38154221bf1c868b99242279" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 45037 } } peers { permanent_uuid: "843989e8da154295a4251b0b8add6d0d" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 35489 } }
I20260806 22:47:36.026777 5828 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ebab2fd38154221bf1c868b99242279 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ebab2fd38154221bf1c868b99242279. Latest consensus state: current_term: 1 leader_uuid: "4ebab2fd38154221bf1c868b99242279" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f85c90dc1cc14b5fa7a2c6635823fde2" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39795 } } peers { permanent_uuid: "4ebab2fd38154221bf1c868b99242279" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 45037 } } peers { permanent_uuid: "843989e8da154295a4251b0b8add6d0d" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 35489 } } }
I20260806 22:47:36.026854 5828 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ebab2fd38154221bf1c868b99242279 [sys.catalog]: This master's current role is: LEADER
I20260806 22:47:36.027338 5864 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:47:36.027609 5864 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:47:36.030395 5761 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 843989e8da154295a4251b0b8add6d0d [term 1 FOLLOWER]: Refusing update from remote peer 4ebab2fd38154221bf1c868b99242279: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:47:36.030939 5828 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4ebab2fd38154221bf1c868b99242279 [LEADER]: Connected to new peer: Peer: permanent_uuid: "843989e8da154295a4251b0b8add6d0d" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 35489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:47:36.033331 5798 sys_catalog.cc:455] T 00000000000000000000000000000000 P 843989e8da154295a4251b0b8add6d0d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ebab2fd38154221bf1c868b99242279. Latest consensus state: current_term: 1 leader_uuid: "4ebab2fd38154221bf1c868b99242279" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f85c90dc1cc14b5fa7a2c6635823fde2" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39795 } } peers { permanent_uuid: "4ebab2fd38154221bf1c868b99242279" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 45037 } } peers { permanent_uuid: "843989e8da154295a4251b0b8add6d0d" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 35489 } } }
I20260806 22:47:36.033473 5798 sys_catalog.cc:458] T 00000000000000000000000000000000 P 843989e8da154295a4251b0b8add6d0d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:47:36.033488 5553 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f85c90dc1cc14b5fa7a2c6635823fde2 [term 1 FOLLOWER]: Refusing update from remote peer 4ebab2fd38154221bf1c868b99242279: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:47:36.033782 5828 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4ebab2fd38154221bf1c868b99242279 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f85c90dc1cc14b5fa7a2c6635823fde2" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:47:36.035328 5845 sys_catalog.cc:455] T 00000000000000000000000000000000 P f85c90dc1cc14b5fa7a2c6635823fde2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ebab2fd38154221bf1c868b99242279. Latest consensus state: current_term: 1 leader_uuid: "4ebab2fd38154221bf1c868b99242279" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f85c90dc1cc14b5fa7a2c6635823fde2" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39795 } } peers { permanent_uuid: "4ebab2fd38154221bf1c868b99242279" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 45037 } } peers { permanent_uuid: "843989e8da154295a4251b0b8add6d0d" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 35489 } } }
I20260806 22:47:36.035451 5845 sys_catalog.cc:458] T 00000000000000000000000000000000 P f85c90dc1cc14b5fa7a2c6635823fde2 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:47:36.039436 5862 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ebab2fd38154221bf1c868b99242279 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4ebab2fd38154221bf1c868b99242279" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f85c90dc1cc14b5fa7a2c6635823fde2" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39795 } } peers { permanent_uuid: "4ebab2fd38154221bf1c868b99242279" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 45037 } } peers { permanent_uuid: "843989e8da154295a4251b0b8add6d0d" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 35489 } } }
I20260806 22:47:36.039559 5862 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ebab2fd38154221bf1c868b99242279 [sys.catalog]: This master's current role is: LEADER
I20260806 22:47:36.040228 5845 sys_catalog.cc:455] T 00000000000000000000000000000000 P f85c90dc1cc14b5fa7a2c6635823fde2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4ebab2fd38154221bf1c868b99242279" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f85c90dc1cc14b5fa7a2c6635823fde2" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39795 } } peers { permanent_uuid: "4ebab2fd38154221bf1c868b99242279" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 45037 } } peers { permanent_uuid: "843989e8da154295a4251b0b8add6d0d" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 35489 } } }
I20260806 22:47:36.040324 5845 sys_catalog.cc:458] T 00000000000000000000000000000000 P f85c90dc1cc14b5fa7a2c6635823fde2 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:47:36.040619 5866 mvcc.cc:205] Tried to move back new op lower bound from 7315687243896799232 to 7315687243884204032. Current Snapshot: MvccSnapshot[applied={T|T < 7315687243896799232}]
I20260806 22:47:36.040773 5870 mvcc.cc:205] Tried to move back new op lower bound from 7315687243896799232 to 7315687243884204032. Current Snapshot: MvccSnapshot[applied={T|T < 7315687243896799232}]
I20260806 22:47:36.041311 5828 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ebab2fd38154221bf1c868b99242279 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4ebab2fd38154221bf1c868b99242279" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f85c90dc1cc14b5fa7a2c6635823fde2" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39795 } } peers { permanent_uuid: "4ebab2fd38154221bf1c868b99242279" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 45037 } } peers { permanent_uuid: "843989e8da154295a4251b0b8add6d0d" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 35489 } } }
I20260806 22:47:36.041399 5828 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ebab2fd38154221bf1c868b99242279 [sys.catalog]: This master's current role is: LEADER
I20260806 22:47:36.041891 5798 sys_catalog.cc:455] T 00000000000000000000000000000000 P 843989e8da154295a4251b0b8add6d0d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4ebab2fd38154221bf1c868b99242279" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f85c90dc1cc14b5fa7a2c6635823fde2" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39795 } } peers { permanent_uuid: "4ebab2fd38154221bf1c868b99242279" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 45037 } } peers { permanent_uuid: "843989e8da154295a4251b0b8add6d0d" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 35489 } } }
I20260806 22:47:36.042001 5798 sys_catalog.cc:458] T 00000000000000000000000000000000 P 843989e8da154295a4251b0b8add6d0d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:47:36.042367 5867 mvcc.cc:205] Tried to move back new op lower bound from 7315687243896799232 to 7315687243884204032. Current Snapshot: MvccSnapshot[applied={T|T < 7315687243896799232}]
I20260806 22:47:36.044003 5864 catalog_manager.cc:1347] Generated new cluster ID: 27dcc0dc958f44df9445ad36cf4ef959
I20260806 22:47:36.044059 5864 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:47:36.058908 5864 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:47:36.059909 5864 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:47:36.065892 5864 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4ebab2fd38154221bf1c868b99242279: Generated new TSK 0
I20260806 22:47:36.066263 5864 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:47:36.901716 5814 catalog_manager.cc:1259] Loaded cluster ID: 27dcc0dc958f44df9445ad36cf4ef959
I20260806 22:47:36.901861 5814 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 843989e8da154295a4251b0b8add6d0d: loading cluster ID for follower catalog manager: success
I20260806 22:47:36.903134 5814 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 843989e8da154295a4251b0b8add6d0d: acquiring CA information for follower catalog manager: success
I20260806 22:47:36.903571 5814 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 843989e8da154295a4251b0b8add6d0d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 22:47:36.937495 5860 catalog_manager.cc:1259] Loaded cluster ID: 27dcc0dc958f44df9445ad36cf4ef959
I20260806 22:47:36.937614 5860 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f85c90dc1cc14b5fa7a2c6635823fde2: loading cluster ID for follower catalog manager: success
I20260806 22:47:36.938556 5860 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f85c90dc1cc14b5fa7a2c6635823fde2: acquiring CA information for follower catalog manager: success
I20260806 22:47:36.938905 5860 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f85c90dc1cc14b5fa7a2c6635823fde2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 22:47:37.125192 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 5487
I20260806 22:47:37.140409 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 5600
I20260806 22:47:37.154987 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 5694
2026-08-06T22:47:37Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3273 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056275803658-18615-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:47:37 dist-test-slave-h3ft krb5kdc[5886](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:47:37 dist-test-slave-h3ft krb5kdc[5886](info): set up 2 sockets
Aug 06 22:47:37 dist-test-slave-h3ft krb5kdc[5886](info): commencing operation
krb5kdc: starting...
W20260806 22:47:38.407936 18615 mini_kdc.cc:121] Time spent starting KDC: real 1.224s user 0.006s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Aug 06 22:47:38 dist-test-slave-h3ft krb5kdc[5886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056458, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:47:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:47:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.45.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab.
Entry for principal kudu/127.18.45.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab.
I20260806 22:47:38.550612 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056275803658-18615-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056275803658-18615-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056275803658-18615-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:45199
--webserver_interface=127.18.45.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:41423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.45.254:45199
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:47:38.644982 5908 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:38.645267 5908 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:38.645308 5908 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:38.646509 5908 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:47:38.646579 5908 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:38.646605 5908 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:38.646637 5908 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:47:38.646687 5908 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:47:38.648329 5908 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:41423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056275803658-18615-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056275803658-18615-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.45.254:45199
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:45199
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.45.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5908
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056275803658-18615-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:47:38.648638 5908 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:38.648918 5908 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:38.654686 5915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:38.654719 5917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:38.654784 5908 server_base.cc:1034] running on GCE node
W20260806 22:47:38.654686 5913 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:38.655272 5908 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:38.655625 5908 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:38.656809 5908 hybrid_clock.cc:648] HybridClock initialized: now 1786056458656792 us; error 53 us; skew 500 ppm
Aug 06 22:47:38 dist-test-slave-h3ft krb5kdc[5886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056458, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:38.659904 5908 init.cc:377] Logged in from keytab as kudu/127.18.45.254@KRBTEST.COM (short username kudu)
I20260806 22:47:38.660463 5908 webserver.cc:460] Webserver started at http://127.18.45.254:37583/ using document root <none> and password file <none>
I20260806 22:47:38.660719 5908 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:38.660764 5908 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:38.660869 5908 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:47:38.662045 5908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056275803658-18615-0/minicluster-data/master-0/data/instance:
uuid: "cba41a491be145569fdb4dde6a953bc4"
format_stamp: "Formatted at 2026-08-06 22:47:38 on dist-test-slave-h3ft"
server_key: "fe8303376f5bce0f4ad5cd2c17da7394"
server_key_iv: "85209e2f92bd9609c951d3b76ea79a27"
server_key_version: "encryptionkey@0"
I20260806 22:47:38.662377 5908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056275803658-18615-0/minicluster-data/master-0/wal/instance:
uuid: "cba41a491be145569fdb4dde6a953bc4"
format_stamp: "Formatted at 2026-08-06 22:47:38 on dist-test-slave-h3ft"
server_key: "fe8303376f5bce0f4ad5cd2c17da7394"
server_key_iv: "85209e2f92bd9609c951d3b76ea79a27"
server_key_version: "encryptionkey@0"
I20260806 22:47:38.663977 5908 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:47:38.664911 5925 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:38.665136 5908 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:47:38.665246 5908 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056275803658-18615-0/minicluster-data/master-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056275803658-18615-0/minicluster-data/master-0/wal
uuid: "cba41a491be145569fdb4dde6a953bc4"
format_stamp: "Formatted at 2026-08-06 22:47:38 on dist-test-slave-h3ft"
server_key: "fe8303376f5bce0f4ad5cd2c17da7394"
server_key_iv: "85209e2f92bd9609c951d3b76ea79a27"
server_key_version: "encryptionkey@0"
I20260806 22:47:38.665333 5908 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056275803658-18615-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056275803658-18615-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056275803658-18615-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:38.681056 5908 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:38.683620 5908 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:38.683823 5908 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:38.688993 5908 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.254:45199
I20260806 22:47:38.689061 6000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.254:45199 every 8 connection(s)
I20260806 22:47:38.689502 5908 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
I20260806 22:47:38.690714 6001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:47:38.694074 6001 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cba41a491be145569fdb4dde6a953bc4: Bootstrap starting.
I20260806 22:47:38.694873 6001 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cba41a491be145569fdb4dde6a953bc4: Neither blocks nor log segments found. Creating new log.
I20260806 22:47:38.695253 6001 log.cc:824] T 00000000000000000000000000000000 P cba41a491be145569fdb4dde6a953bc4: Log is configured to *not* fsync() on all Append() calls
I20260806 22:47:38.696192 6001 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cba41a491be145569fdb4dde6a953bc4: No bootstrap required, opened a new log
I20260806 22:47:38.697242 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 5908
I20260806 22:47:38.697358 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786056275803658-18615-0/minicluster-data/master-0/wal/instance
I20260806 22:47:38.697626 18615 external_mini_cluster.cc:1428] Setting key d4a9291d4571e42560ffe7063df059be
I20260806 22:47:38.698696 6001 raft_consensus.cc:348] T 00000000000000000000000000000000 P cba41a491be145569fdb4dde6a953bc4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cba41a491be145569fdb4dde6a953bc4" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 45199 } }
I20260806 22:47:38.698920 6001 raft_consensus.cc:374] T 00000000000000000000000000000000 P cba41a491be145569fdb4dde6a953bc4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:47:38.698959 6001 raft_consensus.cc:729] T 00000000000000000000000000000000 P cba41a491be145569fdb4dde6a953bc4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cba41a491be145569fdb4dde6a953bc4, State: Initialized, Role: FOLLOWER
I20260806 22:47:38.699174 6001 consensus_queue.cc:260] T 00000000000000000000000000000000 P cba41a491be145569fdb4dde6a953bc4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cba41a491be145569fdb4dde6a953bc4" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 45199 } }
I20260806 22:47:38.699278 6001 raft_consensus.cc:388] T 00000000000000000000000000000000 P cba41a491be145569fdb4dde6a953bc4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:47:38.699326 6001 raft_consensus.cc:482] T 00000000000000000000000000000000 P cba41a491be145569fdb4dde6a953bc4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:47:38.699380 6001 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cba41a491be145569fdb4dde6a953bc4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:47:38.700666 6001 raft_consensus.cc:504] T 00000000000000000000000000000000 P cba41a491be145569fdb4dde6a953bc4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cba41a491be145569fdb4dde6a953bc4" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 45199 } }
I20260806 22:47:38.700790 6001 leader_election.cc:302] T 00000000000000000000000000000000 P cba41a491be145569fdb4dde6a953bc4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cba41a491be145569fdb4dde6a953bc4; no voters:
I20260806 22:47:38.700970 6001 leader_election.cc:288] T 00000000000000000000000000000000 P cba41a491be145569fdb4dde6a953bc4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:47:38.701076 6009 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cba41a491be145569fdb4dde6a953bc4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:47:38.701275 6001 sys_catalog.cc:564] T 00000000000000000000000000000000 P cba41a491be145569fdb4dde6a953bc4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:47:38.701290 6009 raft_consensus.cc:686] T 00000000000000000000000000000000 P cba41a491be145569fdb4dde6a953bc4 [term 1 LEADER]: Becoming Leader. State: Replica: cba41a491be145569fdb4dde6a953bc4, State: Running, Role: LEADER
I20260806 22:47:38.701555 6009 consensus_queue.cc:237] T 00000000000000000000000000000000 P cba41a491be145569fdb4dde6a953bc4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cba41a491be145569fdb4dde6a953bc4" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 45199 } }
I20260806 22:47:38.702018 6010 sys_catalog.cc:455] T 00000000000000000000000000000000 P cba41a491be145569fdb4dde6a953bc4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cba41a491be145569fdb4dde6a953bc4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cba41a491be145569fdb4dde6a953bc4" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 45199 } } }
I20260806 22:47:38.702102 6010 sys_catalog.cc:458] T 00000000000000000000000000000000 P cba41a491be145569fdb4dde6a953bc4 [sys.catalog]: This master's current role is: LEADER
I20260806 22:47:38.702580 6011 sys_catalog.cc:455] T 00000000000000000000000000000000 P cba41a491be145569fdb4dde6a953bc4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cba41a491be145569fdb4dde6a953bc4. Latest consensus state: current_term: 1 leader_uuid: "cba41a491be145569fdb4dde6a953bc4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cba41a491be145569fdb4dde6a953bc4" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 45199 } } }
I20260806 22:47:38.702731 6011 sys_catalog.cc:458] T 00000000000000000000000000000000 P cba41a491be145569fdb4dde6a953bc4 [sys.catalog]: This master's current role is: LEADER
I20260806 22:47:38.702919 6020 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:47:38.703711 6020 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:47:38.705694 6020 catalog_manager.cc:1347] Generated new cluster ID: aa207337b0a343619c2096969f948484
I20260806 22:47:38.705752 6020 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:47:38 dist-test-slave-h3ft krb5kdc[5886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056458, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:47:38.712621 6008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:38.698412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40048 (local address 127.18.45.254:45199)
0806 22:47:38.698671 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:38.698684 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:38.698723 (+ 39us) server_negotiation.cc:407] Connection header received
0806 22:47:38.698860 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:38.698870 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:38.698940 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:38.699142 (+ 202us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:38.699970 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:38.701486 (+ 1516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:38.702743 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:38.703477 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:38.703608 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:38.708372 (+ 4764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:38.708376 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:38.708377 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:38.708393 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:38.710697 (+ 2304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:38.711147 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:38.711151 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:38.711152 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:38.711316 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:38.711532 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:38.711533 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:38.711533 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:38.711905 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:38.712130 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:38.712209 (+ 79us) server_negotiation.cc:299] Negotiation successful
0806 22:47:38.712364 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":131,"threads_started":1}
I20260806 22:47:38.714159 18615 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:47:38.743564 6020 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:47:38.744256 6020 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:47:38.747959 6020 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cba41a491be145569fdb4dde6a953bc4: Generated new TSK 0
I20260806 22:47:38.748204 6020 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:47:38.769237 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 5908
2026-08-06T22:47:38Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1612 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:47:38 dist-test-slave-h3ft krb5kdc[6039](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:47:38 dist-test-slave-h3ft krb5kdc[6039](info): set up 2 sockets
Aug 06 22:47:38 dist-test-slave-h3ft krb5kdc[6039](info): commencing operation
krb5kdc: starting...
W20260806 22:47:40.033732 18615 mini_kdc.cc:121] Time spent starting KDC: real 1.237s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Aug 06 22:47:40 dist-test-slave-h3ft krb5kdc[6039](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056460, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:47:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:47:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.45.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab.
Entry for principal kudu/127.18.45.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab.
I20260806 22:47:40.183295 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:39797
--webserver_interface=127.18.45.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:37755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.45.254:39797
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:47:40.278910 6062 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:40.279282 6062 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:40.279327 6062 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:40.280637 6062 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:47:40.280725 6062 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:40.280754 6062 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:40.280781 6062 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:47:40.280820 6062 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:47:40.282555 6062 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:37755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.45.254:39797
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:39797
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.45.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6062
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:47:40.282913 6062 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:40.283248 6062 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:40.289325 6070 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:40.289480 6062 server_base.cc:1034] running on GCE node
W20260806 22:47:40.289318 6073 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:40.289595 6069 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:40.289942 6062 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:40.290354 6062 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:40.291535 6062 hybrid_clock.cc:648] HybridClock initialized: now 1786056460291515 us; error 46 us; skew 500 ppm
Aug 06 22:47:40 dist-test-slave-h3ft krb5kdc[6039](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056460, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:40.301304 6062 init.cc:377] Logged in from keytab as kudu/127.18.45.254@KRBTEST.COM (short username kudu)
I20260806 22:47:40.301987 6062 webserver.cc:460] Webserver started at http://127.18.45.254:35941/ using document root <none> and password file <none>
I20260806 22:47:40.302299 6062 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:40.302353 6062 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:40.302515 6062 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:47:40.303869 6062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/master-0/data/instance:
uuid: "a6658b79b3ec4a41ab783c6eb6a8f1cc"
format_stamp: "Formatted at 2026-08-06 22:47:40 on dist-test-slave-h3ft"
server_key: "10442fe2d6f5b61f368ac8bed514eec1"
server_key_iv: "2bb5c76b1bbe99b2b3eba2b2052f2fa0"
server_key_version: "encryptionkey@0"
I20260806 22:47:40.304306 6062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/master-0/wal/instance:
uuid: "a6658b79b3ec4a41ab783c6eb6a8f1cc"
format_stamp: "Formatted at 2026-08-06 22:47:40 on dist-test-slave-h3ft"
server_key: "10442fe2d6f5b61f368ac8bed514eec1"
server_key_iv: "2bb5c76b1bbe99b2b3eba2b2052f2fa0"
server_key_version: "encryptionkey@0"
I20260806 22:47:40.305833 6062 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:47:40.306795 6083 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:40.307148 6062 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:47:40.307258 6062 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/master-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/master-0/wal
uuid: "a6658b79b3ec4a41ab783c6eb6a8f1cc"
format_stamp: "Formatted at 2026-08-06 22:47:40 on dist-test-slave-h3ft"
server_key: "10442fe2d6f5b61f368ac8bed514eec1"
server_key_iv: "2bb5c76b1bbe99b2b3eba2b2052f2fa0"
server_key_version: "encryptionkey@0"
I20260806 22:47:40.307335 6062 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:40.329119 6062 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:40.332110 6062 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:40.332322 6062 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:40.338167 6062 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.254:39797
I20260806 22:47:40.338218 6153 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.254:39797 every 8 connection(s)
I20260806 22:47:40.338690 6062 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
I20260806 22:47:40.339967 6154 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:47:40.340745 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 6062
I20260806 22:47:40.340893 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/master-0/wal/instance
I20260806 22:47:40.341176 18615 external_mini_cluster.cc:1428] Setting key 3a6e05c8fcdf9c351ca0e294ff3ec4eb
I20260806 22:47:40.344020 6154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6658b79b3ec4a41ab783c6eb6a8f1cc: Bootstrap starting.
I20260806 22:47:40.344864 6154 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a6658b79b3ec4a41ab783c6eb6a8f1cc: Neither blocks nor log segments found. Creating new log.
I20260806 22:47:40.345265 6154 log.cc:824] T 00000000000000000000000000000000 P a6658b79b3ec4a41ab783c6eb6a8f1cc: Log is configured to *not* fsync() on all Append() calls
I20260806 22:47:40.346189 6154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6658b79b3ec4a41ab783c6eb6a8f1cc: No bootstrap required, opened a new log
I20260806 22:47:40.349093 6154 raft_consensus.cc:348] T 00000000000000000000000000000000 P a6658b79b3ec4a41ab783c6eb6a8f1cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6658b79b3ec4a41ab783c6eb6a8f1cc" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39797 } }
I20260806 22:47:40.349253 6154 raft_consensus.cc:374] T 00000000000000000000000000000000 P a6658b79b3ec4a41ab783c6eb6a8f1cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:47:40.349277 6154 raft_consensus.cc:729] T 00000000000000000000000000000000 P a6658b79b3ec4a41ab783c6eb6a8f1cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6658b79b3ec4a41ab783c6eb6a8f1cc, State: Initialized, Role: FOLLOWER
Aug 06 22:47:40 dist-test-slave-h3ft krb5kdc[6039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056460, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:47:40.349452 6154 consensus_queue.cc:260] T 00000000000000000000000000000000 P a6658b79b3ec4a41ab783c6eb6a8f1cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6658b79b3ec4a41ab783c6eb6a8f1cc" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39797 } }
I20260806 22:47:40.349581 6154 raft_consensus.cc:388] T 00000000000000000000000000000000 P a6658b79b3ec4a41ab783c6eb6a8f1cc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:47:40.349628 6154 raft_consensus.cc:482] T 00000000000000000000000000000000 P a6658b79b3ec4a41ab783c6eb6a8f1cc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:47:40.349686 6154 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a6658b79b3ec4a41ab783c6eb6a8f1cc [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:47:40.350381 6154 raft_consensus.cc:504] T 00000000000000000000000000000000 P a6658b79b3ec4a41ab783c6eb6a8f1cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6658b79b3ec4a41ab783c6eb6a8f1cc" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39797 } }
I20260806 22:47:40.350509 6154 leader_election.cc:302] T 00000000000000000000000000000000 P a6658b79b3ec4a41ab783c6eb6a8f1cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a6658b79b3ec4a41ab783c6eb6a8f1cc; no voters:
I20260806 22:47:40.350708 6154 leader_election.cc:288] T 00000000000000000000000000000000 P a6658b79b3ec4a41ab783c6eb6a8f1cc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:47:40.350752 6159 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a6658b79b3ec4a41ab783c6eb6a8f1cc [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:47:40.350878 6159 raft_consensus.cc:686] T 00000000000000000000000000000000 P a6658b79b3ec4a41ab783c6eb6a8f1cc [term 1 LEADER]: Becoming Leader. State: Replica: a6658b79b3ec4a41ab783c6eb6a8f1cc, State: Running, Role: LEADER
I20260806 22:47:40.351089 6154 sys_catalog.cc:564] T 00000000000000000000000000000000 P a6658b79b3ec4a41ab783c6eb6a8f1cc [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:47:40.351089 6159 consensus_queue.cc:237] T 00000000000000000000000000000000 P a6658b79b3ec4a41ab783c6eb6a8f1cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6658b79b3ec4a41ab783c6eb6a8f1cc" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39797 } }
I20260806 22:47:40.351854 6160 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6658b79b3ec4a41ab783c6eb6a8f1cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a6658b79b3ec4a41ab783c6eb6a8f1cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6658b79b3ec4a41ab783c6eb6a8f1cc" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39797 } } }
I20260806 22:47:40.352015 6160 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6658b79b3ec4a41ab783c6eb6a8f1cc [sys.catalog]: This master's current role is: LEADER
I20260806 22:47:40.352214 6160 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6658b79b3ec4a41ab783c6eb6a8f1cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader a6658b79b3ec4a41ab783c6eb6a8f1cc. Latest consensus state: current_term: 1 leader_uuid: "a6658b79b3ec4a41ab783c6eb6a8f1cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6658b79b3ec4a41ab783c6eb6a8f1cc" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39797 } } }
I20260806 22:47:40.352381 6160 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6658b79b3ec4a41ab783c6eb6a8f1cc [sys.catalog]: This master's current role is: LEADER
I20260806 22:47:40.352828 6174 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:47:40.353526 6174 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:47:40.354615 6157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:40.342030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35658 (local address 127.18.45.254:39797)
0806 22:47:40.342240 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:40.342250 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:40.342281 (+ 31us) server_negotiation.cc:407] Connection header received
0806 22:47:40.342750 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:40.342758 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:40.342814 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:40.342988 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:40.343899 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:40.345080 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:40.346116 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:40.346663 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:40.346763 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:40.350063 (+ 3300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:40.350067 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:40.350067 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:40.350079 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:40.352759 (+ 2680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:40.353197 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:40.353200 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:40.353200 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:40.353323 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:40.353572 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:40.353573 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:40.353574 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:40.353915 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:40.354088 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:40.354230 (+ 142us) server_negotiation.cc:299] Negotiation successful
0806 22:47:40.354381 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":104,"threads_started":1}
I20260806 22:47:40.355598 6174 catalog_manager.cc:1347] Generated new cluster ID: 821df2c33fa14fca8c4df6091d48ed3e
I20260806 22:47:40.355652 6174 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:47:40.361783 6174 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:47:40.362604 6174 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:47:40.365768 6174 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a6658b79b3ec4a41ab783c6eb6a8f1cc: Generated new TSK 0
I20260806 22:47:40.365978 6174 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.45.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab.
Entry for principal kudu/127.18.45.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab.
I20260806 22:47:40.399557 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.193:0
--local_ip_for_outbound_sockets=127.18.45.193
--webserver_interface=127.18.45.193
--webserver_port=0
--tserver_master_addrs=127.18.45.254:39797
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab
--principal=kudu/127.18.45.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:37755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:47:40.494063 6192 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:40.494319 6192 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:40.494346 6192 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:40.495674 6192 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:40.495719 6192 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:40.495747 6192 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.193
I20260806 22:47:40.497434 6192 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:37755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab
--principal=kudu/127.18.45.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.45.193
--webserver_port=0
--tserver_master_addrs=127.18.45.254:39797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6192
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.193
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:47:40.497735 6192 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:40.498035 6192 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:40.504084 6199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:40.504088 6201 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:40.504087 6198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:40.504528 6192 server_base.cc:1034] running on GCE node
I20260806 22:47:40.504767 6192 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:40.505071 6192 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:40.506254 6192 hybrid_clock.cc:648] HybridClock initialized: now 1786056460506236 us; error 35 us; skew 500 ppm
Aug 06 22:47:40 dist-test-slave-h3ft krb5kdc[6039](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056460, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:40.509327 6192 init.cc:377] Logged in from keytab as kudu/127.18.45.193@KRBTEST.COM (short username kudu)
I20260806 22:47:40.509843 6192 webserver.cc:460] Webserver started at http://127.18.45.193:44663/ using document root <none> and password file <none>
I20260806 22:47:40.510164 6192 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:40.510239 6192 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:40.510454 6192 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:47:40.511828 6192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-0/data/instance:
uuid: "3c4699a06f354c3db12e2b056bd7ccfb"
format_stamp: "Formatted at 2026-08-06 22:47:40 on dist-test-slave-h3ft"
server_key: "ab901a5d6a2afc33687d31cb5e9d5235"
server_key_iv: "61bc0e5d62033c27349220873c9abc5d"
server_key_version: "encryptionkey@0"
I20260806 22:47:40.512274 6192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-0/wal/instance:
uuid: "3c4699a06f354c3db12e2b056bd7ccfb"
format_stamp: "Formatted at 2026-08-06 22:47:40 on dist-test-slave-h3ft"
server_key: "ab901a5d6a2afc33687d31cb5e9d5235"
server_key_iv: "61bc0e5d62033c27349220873c9abc5d"
server_key_version: "encryptionkey@0"
I20260806 22:47:40.513922 6192 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 22:47:40.514848 6211 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:40.515098 6192 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:47:40.515230 6192 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-0/wal
uuid: "3c4699a06f354c3db12e2b056bd7ccfb"
format_stamp: "Formatted at 2026-08-06 22:47:40 on dist-test-slave-h3ft"
server_key: "ab901a5d6a2afc33687d31cb5e9d5235"
server_key_iv: "61bc0e5d62033c27349220873c9abc5d"
server_key_version: "encryptionkey@0"
I20260806 22:47:40.515341 6192 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:40.524773 6192 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:40.527144 6192 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:40.527323 6192 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:40.527598 6192 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:47:40.527976 6192 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:47:40.528039 6192 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:40.528085 6192 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:47:40.528115 6192 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:40.535892 6192 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.193:39517
I20260806 22:47:40.535943 6359 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.193:39517 every 8 connection(s)
I20260806 22:47:40.536326 6192 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
Aug 06 22:47:40 dist-test-slave-h3ft krb5kdc[6039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056460, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.193@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:47:40.546492 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 6192
I20260806 22:47:40.546639 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-0/wal/instance
I20260806 22:47:40.546891 18615 external_mini_cluster.cc:1428] Setting key 81ba30774000d61942571be174b7781f
I20260806 22:47:40.546959 6157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:40.537203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:60501 (local address 127.18.45.254:39797)
0806 22:47:40.537277 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:40.537279 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:40.537515 (+ 236us) server_negotiation.cc:407] Connection header received
0806 22:47:40.538198 (+ 683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:40.538199 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:40.538215 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:40.538270 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:40.539462 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:40.540039 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:40.541105 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:40.541335 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:40.541425 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:40.543964 (+ 2539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:40.543966 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:40.543966 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:40.543975 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:40.545370 (+ 1395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:40.545790 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:40.545791 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:40.545791 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:40.545854 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:40.546086 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:40.546086 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:40.546087 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:40.546220 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:40.546277 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:40.546726 (+ 449us) server_negotiation.cc:299] Negotiation successful
0806 22:47:40.546841 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 22:47:40.547516 6362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:40.537164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39797 (local address 127.18.45.193:60501)
0806 22:47:40.537375 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:40.537418 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:40.538070 (+ 652us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:40.538349 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:40.538353 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:40.539112 (+ 759us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:40.539358 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:40.539363 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:40.540144 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:40.540145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:40.541011 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:40.541014 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:40.541437 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:40.541438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:40.541576 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:40.542184 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:40.542194 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:40.543867 (+ 1673us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:40.545449 (+ 1582us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:40.545456 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:40.545458 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:40.545694 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:40.545917 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:40.545918 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:40.545918 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:40.546026 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:40.546295 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:40.546303 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:40.546522 (+ 219us) client_negotiation.cc:769] Sending connection context
0806 22:47:40.547093 (+ 571us) client_negotiation.cc:240] Negotiation successful
0806 22:47:40.547312 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":104,"threads_started":1}
I20260806 22:47:40.548100 6360 heartbeater.cc:347] Connected to a master server at 127.18.45.254:39797
I20260806 22:47:40.548229 6360 heartbeater.cc:464] Registering TS with master...
I20260806 22:47:40.548471 6360 heartbeater.cc:511] Master 127.18.45.254:39797 requested a full tablet report, sending...
I20260806 22:47:40.549090 6108 ts_manager.cc:194] Registered new tserver with Master: 3c4699a06f354c3db12e2b056bd7ccfb (127.18.45.193:39517)
I20260806 22:47:40.549773 6108 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.45.193@KRBTEST.COM'} at 127.18.45.193:60501
WARNING: no policy specified for kudu/127.18.45.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab.
Entry for principal kudu/127.18.45.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab.
I20260806 22:47:40.578764 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.194:0
--local_ip_for_outbound_sockets=127.18.45.194
--webserver_interface=127.18.45.194
--webserver_port=0
--tserver_master_addrs=127.18.45.254:39797
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab
--principal=kudu/127.18.45.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:37755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:47:40.675463 6365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:40.675767 6365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:40.675822 6365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:40.677102 6365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:40.677182 6365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:40.677234 6365 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.194
I20260806 22:47:40.678892 6365 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:37755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab
--principal=kudu/127.18.45.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.45.194
--webserver_port=0
--tserver_master_addrs=127.18.45.254:39797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6365
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.194
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:47:40.679243 6365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:40.679553 6365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:40.685675 6372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:40.685669 6374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:40.685672 6371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:40.686141 6365 server_base.cc:1034] running on GCE node
I20260806 22:47:40.686376 6365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:40.686609 6365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:40.687752 6365 hybrid_clock.cc:648] HybridClock initialized: now 1786056460687742 us; error 37 us; skew 500 ppm
Aug 06 22:47:40 dist-test-slave-h3ft krb5kdc[6039](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056460, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:40.690944 6365 init.cc:377] Logged in from keytab as kudu/127.18.45.194@KRBTEST.COM (short username kudu)
I20260806 22:47:40.691571 6365 webserver.cc:460] Webserver started at http://127.18.45.194:39845/ using document root <none> and password file <none>
I20260806 22:47:40.691792 6365 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:40.691835 6365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:40.691947 6365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:47:40.693136 6365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-1/data/instance:
uuid: "f103722ba11f441dbda1143461876f35"
format_stamp: "Formatted at 2026-08-06 22:47:40 on dist-test-slave-h3ft"
server_key: "92f81981150686adb8d499e315f3f9bb"
server_key_iv: "1b051bfa9ea92fc75a91a60048c980ac"
server_key_version: "encryptionkey@0"
I20260806 22:47:40.693516 6365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-1/wal/instance:
uuid: "f103722ba11f441dbda1143461876f35"
format_stamp: "Formatted at 2026-08-06 22:47:40 on dist-test-slave-h3ft"
server_key: "92f81981150686adb8d499e315f3f9bb"
server_key_iv: "1b051bfa9ea92fc75a91a60048c980ac"
server_key_version: "encryptionkey@0"
I20260806 22:47:40.694959 6365 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260806 22:47:40.695938 6384 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:40.696182 6365 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:47:40.696276 6365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-1/wal
uuid: "f103722ba11f441dbda1143461876f35"
format_stamp: "Formatted at 2026-08-06 22:47:40 on dist-test-slave-h3ft"
server_key: "92f81981150686adb8d499e315f3f9bb"
server_key_iv: "1b051bfa9ea92fc75a91a60048c980ac"
server_key_version: "encryptionkey@0"
I20260806 22:47:40.696339 6365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:40.716323 6365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:40.718971 6365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:40.719187 6365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:40.719441 6365 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:47:40.719820 6365 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:47:40.719858 6365 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:40.719884 6365 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:47:40.719897 6365 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:40.727212 6365 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.194:33871
I20260806 22:47:40.727268 6548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.194:33871 every 8 connection(s)
I20260806 22:47:40.727660 6365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
Aug 06 22:47:40 dist-test-slave-h3ft krb5kdc[6039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056460, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.194@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:47:40.736461 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 6365
I20260806 22:47:40.736609 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-1/wal/instance
I20260806 22:47:40.736847 18615 external_mini_cluster.cc:1428] Setting key b8d233ab3f2cac8792feb3c93fd9d391
I20260806 22:47:40.739068 6157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:40.728528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:33663 (local address 127.18.45.254:39797)
0806 22:47:40.728614 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:40.728617 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:40.728839 (+ 222us) server_negotiation.cc:407] Connection header received
0806 22:47:40.729571 (+ 732us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:40.729573 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:40.729591 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:40.729654 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:40.730805 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:40.731440 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:40.732553 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:40.732774 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:40.732873 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:40.735644 (+ 2771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:40.735646 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:40.735647 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:40.735657 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:40.737350 (+ 1693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:40.737890 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:40.737891 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:40.737892 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:40.737966 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:40.738249 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:40.738251 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:40.738251 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:40.738419 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:40.738492 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:40.738762 (+ 270us) server_negotiation.cc:299] Negotiation successful
0806 22:47:40.738862 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 22:47:40.739135 6551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:40.728507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39797 (local address 127.18.45.194:33663)
0806 22:47:40.728688 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:40.728733 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:40.729429 (+ 696us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:40.729736 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:40.729743 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:40.730430 (+ 687us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:40.730683 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:40.730689 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:40.731549 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:40.731550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:40.732450 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:40.732453 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:40.732882 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:40.732883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:40.733030 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:40.733696 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:40.733707 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:40.735525 (+ 1818us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:40.737441 (+ 1916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:40.737448 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:40.737451 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:40.737801 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:40.738065 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:40.738066 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:40.738066 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:40.738179 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:40.738486 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:40.738491 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:40.738621 (+ 130us) client_negotiation.cc:769] Sending connection context
0806 22:47:40.738713 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 22:47:40.738856 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":87,"threads_started":1}
I20260806 22:47:40.739795 6549 heartbeater.cc:347] Connected to a master server at 127.18.45.254:39797
I20260806 22:47:40.739945 6549 heartbeater.cc:464] Registering TS with master...
I20260806 22:47:40.740170 6549 heartbeater.cc:511] Master 127.18.45.254:39797 requested a full tablet report, sending...
I20260806 22:47:40.740677 6108 ts_manager.cc:194] Registered new tserver with Master: f103722ba11f441dbda1143461876f35 (127.18.45.194:33871)
I20260806 22:47:40.741096 6108 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.45.194@KRBTEST.COM'} at 127.18.45.194:33663
WARNING: no policy specified for kudu/127.18.45.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab.
Entry for principal kudu/127.18.45.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab.
I20260806 22:47:40.769878 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.195:0
--local_ip_for_outbound_sockets=127.18.45.195
--webserver_interface=127.18.45.195
--webserver_port=0
--tserver_master_addrs=127.18.45.254:39797
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab
--principal=kudu/127.18.45.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:37755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:47:40.867719 6554 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:40.868012 6554 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:40.868055 6554 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:40.869274 6554 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:40.869346 6554 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:40.869395 6554 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.195
I20260806 22:47:40.871361 6554 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:37755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab
--principal=kudu/127.18.45.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.45.195
--webserver_port=0
--tserver_master_addrs=127.18.45.254:39797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6554
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.195
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:47:40.871671 6554 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:40.871958 6554 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:40.877743 6559 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:40.877743 6562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:40.877777 6560 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:40.878095 6554 server_base.cc:1034] running on GCE node
I20260806 22:47:40.878365 6554 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:40.878598 6554 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:40.879750 6554 hybrid_clock.cc:648] HybridClock initialized: now 1786056460879738 us; error 33 us; skew 500 ppm
Aug 06 22:47:40 dist-test-slave-h3ft krb5kdc[6039](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056460, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:40.882619 6554 init.cc:377] Logged in from keytab as kudu/127.18.45.195@KRBTEST.COM (short username kudu)
I20260806 22:47:40.883268 6554 webserver.cc:460] Webserver started at http://127.18.45.195:44609/ using document root <none> and password file <none>
I20260806 22:47:40.883565 6554 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:40.883618 6554 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:40.883765 6554 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:47:40.884925 6554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-2/data/instance:
uuid: "970e2afa7c324e03aa24b28063eae728"
format_stamp: "Formatted at 2026-08-06 22:47:40 on dist-test-slave-h3ft"
server_key: "1b0acecf3389179e2102ca862d630c30"
server_key_iv: "6cb2d268c209228a8f4b607284fa7c66"
server_key_version: "encryptionkey@0"
I20260806 22:47:40.885274 6554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-2/wal/instance:
uuid: "970e2afa7c324e03aa24b28063eae728"
format_stamp: "Formatted at 2026-08-06 22:47:40 on dist-test-slave-h3ft"
server_key: "1b0acecf3389179e2102ca862d630c30"
server_key_iv: "6cb2d268c209228a8f4b607284fa7c66"
server_key_version: "encryptionkey@0"
I20260806 22:47:40.886694 6554 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:47:40.887526 6570 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:40.887742 6554 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:47:40.887825 6554 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-2/wal
uuid: "970e2afa7c324e03aa24b28063eae728"
format_stamp: "Formatted at 2026-08-06 22:47:40 on dist-test-slave-h3ft"
server_key: "1b0acecf3389179e2102ca862d630c30"
server_key_iv: "6cb2d268c209228a8f4b607284fa7c66"
server_key_version: "encryptionkey@0"
I20260806 22:47:40.887881 6554 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:40.898259 6554 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:40.900794 6554 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:40.900924 6554 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:40.901146 6554 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:47:40.901528 6554 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:47:40.901566 6554 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:40.901592 6554 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:47:40.901604 6554 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:40.909154 6554 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.195:36453
I20260806 22:47:40.909206 6718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.195:36453 every 8 connection(s)
I20260806 22:47:40.909596 6554 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
I20260806 22:47:40.916519 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 6554
I20260806 22:47:40.916610 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-2/wal/instance
I20260806 22:47:40.916810 18615 external_mini_cluster.cc:1428] Setting key 3120e4e519a33db40b28e0ac0749261a
Aug 06 22:47:40 dist-test-slave-h3ft krb5kdc[6039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056460, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.195@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:47:40.921109 6157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:40.910554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.195:43563 (local address 127.18.45.254:39797)
0806 22:47:40.910652 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:40.910655 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:40.910865 (+ 210us) server_negotiation.cc:407] Connection header received
0806 22:47:40.911547 (+ 682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:40.911564 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:40.911600 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:40.911681 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:40.912883 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:40.913466 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:40.914837 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:40.915091 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:40.915152 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:40.917904 (+ 2752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:40.917906 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:40.917907 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:40.917916 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:40.919623 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:40.920116 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:40.920117 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:40.920117 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:40.920178 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:40.920422 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:40.920423 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:40.920423 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:40.920537 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:40.920592 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:40.920956 (+ 364us) server_negotiation.cc:299] Negotiation successful
0806 22:47:40.921030 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 22:47:40.921427 6721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:40.910512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39797 (local address 127.18.45.195:43563)
0806 22:47:40.910706 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:40.910756 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:40.911438 (+ 682us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:40.911807 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:40.911811 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:40.912525 (+ 714us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:40.912783 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:40.912790 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:40.913581 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:40.913582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:40.914750 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:40.914754 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:40.915176 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:40.915178 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:40.915304 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:40.915979 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:40.915989 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:40.917784 (+ 1795us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:40.919721 (+ 1937us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:40.919728 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:40.919730 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:40.920031 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:40.920239 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:40.920240 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:40.920240 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:40.920365 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:40.920644 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:40.920652 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:40.920813 (+ 161us) client_negotiation.cc:769] Sending connection context
0806 22:47:40.920930 (+ 117us) client_negotiation.cc:240] Negotiation successful
0806 22:47:40.921087 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":106,"threads_started":1}
I20260806 22:47:40.921960 6719 heartbeater.cc:347] Connected to a master server at 127.18.45.254:39797
I20260806 22:47:40.922093 6719 heartbeater.cc:464] Registering TS with master...
I20260806 22:47:40.922325 6719 heartbeater.cc:511] Master 127.18.45.254:39797 requested a full tablet report, sending...
I20260806 22:47:40.922811 6108 ts_manager.cc:194] Registered new tserver with Master: 970e2afa7c324e03aa24b28063eae728 (127.18.45.195:36453)
I20260806 22:47:40.923283 6108 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.45.195@KRBTEST.COM'} at 127.18.45.195:43563
I20260806 22:47:40.929013 18615 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:47:40.936753 6157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:40.930649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35664 (local address 127.18.45.254:39797)
0806 22:47:40.930723 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:40.930725 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:40.930796 (+ 71us) server_negotiation.cc:407] Connection header received
0806 22:47:40.930841 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:40.930842 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:40.930851 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:40.930897 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:40.931532 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:40.932024 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:40.932781 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:40.932961 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:40.933018 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:40.934301 (+ 1283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:40.934302 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:40.934302 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:40.934308 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:40.935649 (+ 1341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:40.935993 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:40.935994 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:40.935995 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:40.936047 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:40.936217 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:40.936217 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:40.936217 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:40.936388 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:40.936467 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:40.936615 (+ 148us) server_negotiation.cc:299] Negotiation successful
0806 22:47:40.936677 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 22:47:40.938843 6108 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35664:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:47:40.941047 6108 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:47:40.949280 6735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:40.944150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.195:36453 (local address 127.0.0.1:43490)
0806 22:47:40.944363 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:40.944417 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:40.944498 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:40.944935 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:40.944940 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:40.944943 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:47:40.945040 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:40.945042 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:40.946704 (+ 1662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:40.946705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:40.948150 (+ 1445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:40.948152 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:40.948926 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:40.948928 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:40.949015 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:40.949033 (+ 18us) client_negotiation.cc:769] Sending connection context
0806 22:47:40.949117 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 22:47:40.949184 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"mutex_wait_us":95,"thread_start_us":81,"threads_started":1}
I20260806 22:47:40.949294 6738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:40.944270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43490 (local address 127.18.45.195:36453)
0806 22:47:40.944705 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:40.944709 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:40.944722 (+ 13us) server_negotiation.cc:407] Connection header received
0806 22:47:40.944777 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:40.944780 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:40.944801 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:40.944915 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:47:40.945147 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:40.946570 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:40.948265 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:40.948809 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:40.948925 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:40.949005 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:40.949135 (+ 130us) server_negotiation.cc:299] Negotiation successful
0806 22:47:40.949190 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":126,"threads_started":1}
I20260806 22:47:40.949600 6740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:40.944212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.194:33871 (local address 127.0.0.1:54336)
0806 22:47:40.944646 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:40.944660 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:40.944734 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:40.945175 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:40.945176 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:40.945179 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:47:40.945251 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:40.945252 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:40.946761 (+ 1509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:40.946762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:40.947950 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:40.947953 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:40.949045 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:40.949047 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:40.949152 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:40.949157 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:47:40.949211 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 22:47:40.949265 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":120,"threads_started":1}
I20260806 22:47:40.949654 6739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:40.944390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54336 (local address 127.18.45.194:33871)
0806 22:47:40.944621 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:40.944627 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:40.944801 (+ 174us) server_negotiation.cc:407] Connection header received
0806 22:47:40.944857 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:40.944859 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:40.944896 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:40.944985 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:47:40.945339 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:40.946640 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:40.948052 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:40.948862 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:40.948942 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:40.949005 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:40.949357 (+ 352us) server_negotiation.cc:299] Negotiation successful
0806 22:47:40.949561 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":119,"threads_started":1}
I20260806 22:47:40.950313 6634 tablet_service.cc:1467] Processing CreateTablet for tablet fe72ad4a750742b0a11ac3931c5eb63d (DEFAULT_TABLE table=test-table [id=5f7d3059196f47f8b51a70ebd4d29d59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:47:40.950402 6458 tablet_service.cc:1467] Processing CreateTablet for tablet fe72ad4a750742b0a11ac3931c5eb63d (DEFAULT_TABLE table=test-table [id=5f7d3059196f47f8b51a70ebd4d29d59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:47:40.950774 6634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe72ad4a750742b0a11ac3931c5eb63d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:47:40.950778 6458 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe72ad4a750742b0a11ac3931c5eb63d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:47:40.954574 6743 tablet_bootstrap.cc:492] T fe72ad4a750742b0a11ac3931c5eb63d P f103722ba11f441dbda1143461876f35: Bootstrap starting.
I20260806 22:47:40.954804 6737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:40.944472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.193:39517 (local address 127.0.0.1:59494)
0806 22:47:40.945186 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:40.945197 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:40.945261 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:40.945667 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:40.945668 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:40.945671 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:47:40.945769 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:40.945771 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:40.946752 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:40.946753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:40.948142 (+ 1389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:40.948145 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:40.949261 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:40.949262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:40.949344 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:40.949349 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:47:40.954612 (+ 5263us) client_negotiation.cc:240] Negotiation successful
0806 22:47:40.954690 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":686,"mutex_wait_us":83,"thread_start_us":100,"threads_started":1}
I20260806 22:47:40.955164 6741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:40.944809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59494 (local address 127.18.45.193:39517)
0806 22:47:40.945109 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:40.945113 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:40.945326 (+ 213us) server_negotiation.cc:407] Connection header received
0806 22:47:40.945386 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:40.945389 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:40.945408 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:40.945499 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:47:40.945851 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:40.946599 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:40.948246 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:40.949089 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:40.949163 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:40.949225 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:40.954955 (+ 5730us) server_negotiation.cc:299] Negotiation successful
0806 22:47:40.955052 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":122,"threads_started":1}
I20260806 22:47:40.955545 6743 tablet_bootstrap.cc:654] T fe72ad4a750742b0a11ac3931c5eb63d P f103722ba11f441dbda1143461876f35: Neither blocks nor log segments found. Creating new log.
I20260806 22:47:40.956030 6276 tablet_service.cc:1467] Processing CreateTablet for tablet fe72ad4a750742b0a11ac3931c5eb63d (DEFAULT_TABLE table=test-table [id=5f7d3059196f47f8b51a70ebd4d29d59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:47:40.956221 6743 log.cc:824] T fe72ad4a750742b0a11ac3931c5eb63d P f103722ba11f441dbda1143461876f35: Log is configured to *not* fsync() on all Append() calls
I20260806 22:47:40.956488 6276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe72ad4a750742b0a11ac3931c5eb63d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:47:40.956741 6742 tablet_bootstrap.cc:492] T fe72ad4a750742b0a11ac3931c5eb63d P 970e2afa7c324e03aa24b28063eae728: Bootstrap starting.
I20260806 22:47:40.957388 6742 tablet_bootstrap.cc:654] T fe72ad4a750742b0a11ac3931c5eb63d P 970e2afa7c324e03aa24b28063eae728: Neither blocks nor log segments found. Creating new log.
I20260806 22:47:40.957989 6742 log.cc:824] T fe72ad4a750742b0a11ac3931c5eb63d P 970e2afa7c324e03aa24b28063eae728: Log is configured to *not* fsync() on all Append() calls
I20260806 22:47:40.958600 6743 tablet_bootstrap.cc:492] T fe72ad4a750742b0a11ac3931c5eb63d P f103722ba11f441dbda1143461876f35: No bootstrap required, opened a new log
I20260806 22:47:40.958689 6743 ts_tablet_manager.cc:1397] T fe72ad4a750742b0a11ac3931c5eb63d P f103722ba11f441dbda1143461876f35: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 22:47:40.958909 6742 tablet_bootstrap.cc:492] T fe72ad4a750742b0a11ac3931c5eb63d P 970e2afa7c324e03aa24b28063eae728: No bootstrap required, opened a new log
I20260806 22:47:40.958984 6742 ts_tablet_manager.cc:1397] T fe72ad4a750742b0a11ac3931c5eb63d P 970e2afa7c324e03aa24b28063eae728: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:47:40.960093 6747 tablet_bootstrap.cc:492] T fe72ad4a750742b0a11ac3931c5eb63d P 3c4699a06f354c3db12e2b056bd7ccfb: Bootstrap starting.
I20260806 22:47:40.960981 6747 tablet_bootstrap.cc:654] T fe72ad4a750742b0a11ac3931c5eb63d P 3c4699a06f354c3db12e2b056bd7ccfb: Neither blocks nor log segments found. Creating new log.
I20260806 22:47:40.961227 6743 raft_consensus.cc:348] T fe72ad4a750742b0a11ac3931c5eb63d P f103722ba11f441dbda1143461876f35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "970e2afa7c324e03aa24b28063eae728" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 36453 } } peers { permanent_uuid: "f103722ba11f441dbda1143461876f35" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33871 } } peers { permanent_uuid: "3c4699a06f354c3db12e2b056bd7ccfb" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 39517 } }
I20260806 22:47:40.961298 6742 raft_consensus.cc:348] T fe72ad4a750742b0a11ac3931c5eb63d P 970e2afa7c324e03aa24b28063eae728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "970e2afa7c324e03aa24b28063eae728" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 36453 } } peers { permanent_uuid: "f103722ba11f441dbda1143461876f35" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33871 } } peers { permanent_uuid: "3c4699a06f354c3db12e2b056bd7ccfb" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 39517 } }
I20260806 22:47:40.961452 6742 raft_consensus.cc:374] T fe72ad4a750742b0a11ac3931c5eb63d P 970e2afa7c324e03aa24b28063eae728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:47:40.961452 6743 raft_consensus.cc:374] T fe72ad4a750742b0a11ac3931c5eb63d P f103722ba11f441dbda1143461876f35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:47:40.961494 6743 raft_consensus.cc:729] T fe72ad4a750742b0a11ac3931c5eb63d P f103722ba11f441dbda1143461876f35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f103722ba11f441dbda1143461876f35, State: Initialized, Role: FOLLOWER
I20260806 22:47:40.961494 6742 raft_consensus.cc:729] T fe72ad4a750742b0a11ac3931c5eb63d P 970e2afa7c324e03aa24b28063eae728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 970e2afa7c324e03aa24b28063eae728, State: Initialized, Role: FOLLOWER
I20260806 22:47:40.961645 6742 consensus_queue.cc:260] T fe72ad4a750742b0a11ac3931c5eb63d P 970e2afa7c324e03aa24b28063eae728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "970e2afa7c324e03aa24b28063eae728" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 36453 } } peers { permanent_uuid: "f103722ba11f441dbda1143461876f35" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33871 } } peers { permanent_uuid: "3c4699a06f354c3db12e2b056bd7ccfb" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 39517 } }
I20260806 22:47:40.961683 6747 log.cc:824] T fe72ad4a750742b0a11ac3931c5eb63d P 3c4699a06f354c3db12e2b056bd7ccfb: Log is configured to *not* fsync() on all Append() calls
I20260806 22:47:40.961675 6743 consensus_queue.cc:260] T fe72ad4a750742b0a11ac3931c5eb63d P f103722ba11f441dbda1143461876f35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "970e2afa7c324e03aa24b28063eae728" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 36453 } } peers { permanent_uuid: "f103722ba11f441dbda1143461876f35" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33871 } } peers { permanent_uuid: "3c4699a06f354c3db12e2b056bd7ccfb" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 39517 } }
I20260806 22:47:40.962013 6742 ts_tablet_manager.cc:1428] T fe72ad4a750742b0a11ac3931c5eb63d P 970e2afa7c324e03aa24b28063eae728: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:47:40.962013 6743 ts_tablet_manager.cc:1428] T fe72ad4a750742b0a11ac3931c5eb63d P f103722ba11f441dbda1143461876f35: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:47:40.962178 6719 heartbeater.cc:503] Master 127.18.45.254:39797 was elected leader, sending a full tablet report...
I20260806 22:47:40.962282 6549 heartbeater.cc:503] Master 127.18.45.254:39797 was elected leader, sending a full tablet report...
I20260806 22:47:40.963091 6747 tablet_bootstrap.cc:492] T fe72ad4a750742b0a11ac3931c5eb63d P 3c4699a06f354c3db12e2b056bd7ccfb: No bootstrap required, opened a new log
I20260806 22:47:40.963193 6747 ts_tablet_manager.cc:1397] T fe72ad4a750742b0a11ac3931c5eb63d P 3c4699a06f354c3db12e2b056bd7ccfb: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:47:40.965030 6747 raft_consensus.cc:348] T fe72ad4a750742b0a11ac3931c5eb63d P 3c4699a06f354c3db12e2b056bd7ccfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "970e2afa7c324e03aa24b28063eae728" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 36453 } } peers { permanent_uuid: "f103722ba11f441dbda1143461876f35" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33871 } } peers { permanent_uuid: "3c4699a06f354c3db12e2b056bd7ccfb" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 39517 } }
I20260806 22:47:40.965173 6747 raft_consensus.cc:374] T fe72ad4a750742b0a11ac3931c5eb63d P 3c4699a06f354c3db12e2b056bd7ccfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:47:40.965197 6747 raft_consensus.cc:729] T fe72ad4a750742b0a11ac3931c5eb63d P 3c4699a06f354c3db12e2b056bd7ccfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c4699a06f354c3db12e2b056bd7ccfb, State: Initialized, Role: FOLLOWER
I20260806 22:47:40.965337 6747 consensus_queue.cc:260] T fe72ad4a750742b0a11ac3931c5eb63d P 3c4699a06f354c3db12e2b056bd7ccfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "970e2afa7c324e03aa24b28063eae728" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 36453 } } peers { permanent_uuid: "f103722ba11f441dbda1143461876f35" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33871 } } peers { permanent_uuid: "3c4699a06f354c3db12e2b056bd7ccfb" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 39517 } }
I20260806 22:47:40.965664 6747 ts_tablet_manager.cc:1428] T fe72ad4a750742b0a11ac3931c5eb63d P 3c4699a06f354c3db12e2b056bd7ccfb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:47:40.965668 6360 heartbeater.cc:503] Master 127.18.45.254:39797 was elected leader, sending a full tablet report...
W20260806 22:47:40.978274 6550 tablet.cc:2369] T fe72ad4a750742b0a11ac3931c5eb63d P f103722ba11f441dbda1143461876f35: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:47:41.037302 6361 tablet.cc:2369] T fe72ad4a750742b0a11ac3931c5eb63d P 3c4699a06f354c3db12e2b056bd7ccfb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:47:41.160409 6720 tablet.cc:2369] T fe72ad4a750742b0a11ac3931c5eb63d P 970e2afa7c324e03aa24b28063eae728: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:47:41.363716 6751 raft_consensus.cc:482] T fe72ad4a750742b0a11ac3931c5eb63d P 3c4699a06f354c3db12e2b056bd7ccfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:47:41.363960 6751 raft_consensus.cc:504] T fe72ad4a750742b0a11ac3931c5eb63d P 3c4699a06f354c3db12e2b056bd7ccfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "970e2afa7c324e03aa24b28063eae728" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 36453 } } peers { permanent_uuid: "f103722ba11f441dbda1143461876f35" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33871 } } peers { permanent_uuid: "3c4699a06f354c3db12e2b056bd7ccfb" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 39517 } }
I20260806 22:47:41.364583 6751 leader_election.cc:288] T fe72ad4a750742b0a11ac3931c5eb63d P 3c4699a06f354c3db12e2b056bd7ccfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 970e2afa7c324e03aa24b28063eae728 (127.18.45.195:36453), f103722ba11f441dbda1143461876f35 (127.18.45.194:33871)
I20260806 22:47:41.368357 6739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:41.365078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:37953 (local address 127.18.45.194:33871)
0806 22:47:41.365140 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:41.365143 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:41.365199 (+ 56us) server_negotiation.cc:407] Connection header received
0806 22:47:41.365336 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:41.365338 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:41.365362 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:41.365441 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:47:41.365697 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:41.366234 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:41.367505 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:41.367919 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:41.367981 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:41.367992 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:41.368213 (+ 221us) server_negotiation.cc:299] Negotiation successful
0806 22:47:41.368274 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 22:47:41.368345 6752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:41.364930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.194:33871 (local address 127.18.45.193:37953)
0806 22:47:41.365112 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:41.365128 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:41.365218 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:41.365448 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:41.365449 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:41.365452 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:47:41.365607 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:41.365610 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:41.366335 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:41.366337 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:41.367400 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:41.367403 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:41.368036 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:41.368038 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:41.368118 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:41.368124 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 22:47:41.368179 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 22:47:41.368216 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":72,"thread_start_us":87,"threads_started":1}
I20260806 22:47:41.368425 6738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:41.365145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:49361 (local address 127.18.45.195:36453)
0806 22:47:41.365207 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:41.365209 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:41.365222 (+ 13us) server_negotiation.cc:407] Connection header received
0806 22:47:41.365344 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:41.365345 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:41.365356 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:41.365439 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:47:41.365725 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:41.366234 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:41.367556 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:41.368011 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:41.368092 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:41.368105 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:41.368282 (+ 177us) server_negotiation.cc:299] Negotiation successful
0806 22:47:41.368331 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 22:47:41.368480 6753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:41.364897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.195:36453 (local address 127.18.45.193:49361)
0806 22:47:41.365111 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:41.365128 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:41.365220 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:41.365452 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:41.365453 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:41.365455 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:47:41.365616 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:41.365618 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:41.366331 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:41.366332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:41.367406 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:41.367409 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:41.368107 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:41.368108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:41.368190 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:41.368198 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 22:47:41.368258 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 22:47:41.368298 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"mutex_wait_us":141,"thread_start_us":115,"threads_started":1}
I20260806 22:47:41.368988 6487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe72ad4a750742b0a11ac3931c5eb63d" candidate_uuid: "3c4699a06f354c3db12e2b056bd7ccfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f103722ba11f441dbda1143461876f35" is_pre_election: true
I20260806 22:47:41.368963 6659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe72ad4a750742b0a11ac3931c5eb63d" candidate_uuid: "3c4699a06f354c3db12e2b056bd7ccfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "970e2afa7c324e03aa24b28063eae728" is_pre_election: true
I20260806 22:47:41.369177 6487 raft_consensus.cc:2454] T fe72ad4a750742b0a11ac3931c5eb63d P f103722ba11f441dbda1143461876f35 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c4699a06f354c3db12e2b056bd7ccfb in term 0.
I20260806 22:47:41.369190 6659 raft_consensus.cc:2454] T fe72ad4a750742b0a11ac3931c5eb63d P 970e2afa7c324e03aa24b28063eae728 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c4699a06f354c3db12e2b056bd7ccfb in term 0.
I20260806 22:47:41.369472 6214 leader_election.cc:302] T fe72ad4a750742b0a11ac3931c5eb63d P 3c4699a06f354c3db12e2b056bd7ccfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c4699a06f354c3db12e2b056bd7ccfb, f103722ba11f441dbda1143461876f35; no voters:
I20260806 22:47:41.369648 6751 raft_consensus.cc:2781] T fe72ad4a750742b0a11ac3931c5eb63d P 3c4699a06f354c3db12e2b056bd7ccfb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:47:41.369721 6751 raft_consensus.cc:482] T fe72ad4a750742b0a11ac3931c5eb63d P 3c4699a06f354c3db12e2b056bd7ccfb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:47:41.369740 6751 raft_consensus.cc:3037] T fe72ad4a750742b0a11ac3931c5eb63d P 3c4699a06f354c3db12e2b056bd7ccfb [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:47:41.370707 6751 raft_consensus.cc:504] T fe72ad4a750742b0a11ac3931c5eb63d P 3c4699a06f354c3db12e2b056bd7ccfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "970e2afa7c324e03aa24b28063eae728" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 36453 } } peers { permanent_uuid: "f103722ba11f441dbda1143461876f35" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33871 } } peers { permanent_uuid: "3c4699a06f354c3db12e2b056bd7ccfb" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 39517 } }
I20260806 22:47:41.370877 6751 leader_election.cc:288] T fe72ad4a750742b0a11ac3931c5eb63d P 3c4699a06f354c3db12e2b056bd7ccfb [CANDIDATE]: Term 1 election: Requested vote from peers 970e2afa7c324e03aa24b28063eae728 (127.18.45.195:36453), f103722ba11f441dbda1143461876f35 (127.18.45.194:33871)
I20260806 22:47:41.371104 6487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe72ad4a750742b0a11ac3931c5eb63d" candidate_uuid: "3c4699a06f354c3db12e2b056bd7ccfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f103722ba11f441dbda1143461876f35"
I20260806 22:47:41.371112 6659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe72ad4a750742b0a11ac3931c5eb63d" candidate_uuid: "3c4699a06f354c3db12e2b056bd7ccfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "970e2afa7c324e03aa24b28063eae728"
I20260806 22:47:41.371218 6659 raft_consensus.cc:3037] T fe72ad4a750742b0a11ac3931c5eb63d P 970e2afa7c324e03aa24b28063eae728 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:47:41.371225 6487 raft_consensus.cc:3037] T fe72ad4a750742b0a11ac3931c5eb63d P f103722ba11f441dbda1143461876f35 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:47:41.372149 6487 raft_consensus.cc:2454] T fe72ad4a750742b0a11ac3931c5eb63d P f103722ba11f441dbda1143461876f35 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c4699a06f354c3db12e2b056bd7ccfb in term 1.
I20260806 22:47:41.372200 6659 raft_consensus.cc:2454] T fe72ad4a750742b0a11ac3931c5eb63d P 970e2afa7c324e03aa24b28063eae728 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c4699a06f354c3db12e2b056bd7ccfb in term 1.
I20260806 22:47:41.372364 6214 leader_election.cc:302] T fe72ad4a750742b0a11ac3931c5eb63d P 3c4699a06f354c3db12e2b056bd7ccfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c4699a06f354c3db12e2b056bd7ccfb, f103722ba11f441dbda1143461876f35; no voters:
I20260806 22:47:41.372483 6751 raft_consensus.cc:2781] T fe72ad4a750742b0a11ac3931c5eb63d P 3c4699a06f354c3db12e2b056bd7ccfb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:47:41.372756 6751 raft_consensus.cc:686] T fe72ad4a750742b0a11ac3931c5eb63d P 3c4699a06f354c3db12e2b056bd7ccfb [term 1 LEADER]: Becoming Leader. State: Replica: 3c4699a06f354c3db12e2b056bd7ccfb, State: Running, Role: LEADER
I20260806 22:47:41.372881 6751 consensus_queue.cc:237] T fe72ad4a750742b0a11ac3931c5eb63d P 3c4699a06f354c3db12e2b056bd7ccfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "970e2afa7c324e03aa24b28063eae728" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 36453 } } peers { permanent_uuid: "f103722ba11f441dbda1143461876f35" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33871 } } peers { permanent_uuid: "3c4699a06f354c3db12e2b056bd7ccfb" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 39517 } }
I20260806 22:47:41.373945 6108 catalog_manager.cc:5495] T fe72ad4a750742b0a11ac3931c5eb63d P 3c4699a06f354c3db12e2b056bd7ccfb reported cstate change: term changed from 0 to 1, leader changed from <none> to 3c4699a06f354c3db12e2b056bd7ccfb (127.18.45.193). New cstate: current_term: 1 leader_uuid: "3c4699a06f354c3db12e2b056bd7ccfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "970e2afa7c324e03aa24b28063eae728" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 36453 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f103722ba11f441dbda1143461876f35" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c4699a06f354c3db12e2b056bd7ccfb" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 39517 } health_report { overall_health: HEALTHY } } }
I20260806 22:47:41.391980 6741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:41.388854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59500 (local address 127.18.45.193:39517)
0806 22:47:41.388933 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:41.388935 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:41.388945 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:47:41.389004 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:41.389005 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:41.389031 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:41.389097 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:47:41.389258 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:41.389794 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:41.390768 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:41.391183 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:41.391279 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:41.391478 (+ 199us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:47:41.391659 (+ 181us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:47:41.391759 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:41.391815 (+ 56us) server_negotiation.cc:299] Negotiation successful
0806 22:47:41.391844 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 22:47:41.394449 6659 raft_consensus.cc:1261] T fe72ad4a750742b0a11ac3931c5eb63d P 970e2afa7c324e03aa24b28063eae728 [term 1 FOLLOWER]: Refusing update from remote peer 3c4699a06f354c3db12e2b056bd7ccfb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:47:41.394507 6487 raft_consensus.cc:1261] T fe72ad4a750742b0a11ac3931c5eb63d P f103722ba11f441dbda1143461876f35 [term 1 FOLLOWER]: Refusing update from remote peer 3c4699a06f354c3db12e2b056bd7ccfb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:47:41.394948 6751 consensus_queue.cc:1035] T fe72ad4a750742b0a11ac3931c5eb63d P 3c4699a06f354c3db12e2b056bd7ccfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "f103722ba11f441dbda1143461876f35" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:47:41.395197 6754 consensus_queue.cc:1035] T fe72ad4a750742b0a11ac3931c5eb63d P 3c4699a06f354c3db12e2b056bd7ccfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "970e2afa7c324e03aa24b28063eae728" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 36453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:47:41.397960 6759 mvcc.cc:205] Tried to move back new op lower bound from 7315687265867886592 to 7315687265783939072. Current Snapshot: MvccSnapshot[applied={T|T < 7315687265867886592}]
I20260806 22:47:41.398434 6762 mvcc.cc:205] Tried to move back new op lower bound from 7315687265867886592 to 7315687265783939072. Current Snapshot: MvccSnapshot[applied={T|T < 7315687265867886592}]
I20260806 22:47:41.398627 6761 mvcc.cc:205] Tried to move back new op lower bound from 7315687265867886592 to 7315687265783939072. Current Snapshot: MvccSnapshot[applied={T|T < 7315687265867886592}]
I20260806 22:47:41.401275 6107 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35664:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:47:41.401384 6107 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35664:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:47:41.402359 6107 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a6658b79b3ec4a41ab783c6eb6a8f1cc: Sending DeleteTablet for 3 replicas of tablet fe72ad4a750742b0a11ac3931c5eb63d
I20260806 22:47:41.402755 6634 tablet_service.cc:1514] Processing DeleteTablet for tablet fe72ad4a750742b0a11ac3931c5eb63d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:47:41 UTC) from {username='kudu', principal='kudu/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:43490
I20260806 22:47:41.402822 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 6192
I20260806 22:47:41.402792 6458 tablet_service.cc:1514] Processing DeleteTablet for tablet fe72ad4a750742b0a11ac3931c5eb63d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:47:41 UTC) from {username='kudu', principal='kudu/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:54336
I20260806 22:47:41.402861 6276 tablet_service.cc:1514] Processing DeleteTablet for tablet fe72ad4a750742b0a11ac3931c5eb63d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:47:41 UTC) from {username='kudu', principal='kudu/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:59494
I20260806 22:47:41.411275 6773 tablet_replica.cc:330] T fe72ad4a750742b0a11ac3931c5eb63d P f103722ba11f441dbda1143461876f35: stopping tablet replica
I20260806 22:47:41.411612 6772 tablet_replica.cc:330] T fe72ad4a750742b0a11ac3931c5eb63d P 970e2afa7c324e03aa24b28063eae728: stopping tablet replica
I20260806 22:47:41.411731 6772 raft_consensus.cc:2229] T fe72ad4a750742b0a11ac3931c5eb63d P 970e2afa7c324e03aa24b28063eae728 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:47:41.411922 6772 raft_consensus.cc:2258] T fe72ad4a750742b0a11ac3931c5eb63d P 970e2afa7c324e03aa24b28063eae728 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:47:41.412145 6773 raft_consensus.cc:2229] T fe72ad4a750742b0a11ac3931c5eb63d P f103722ba11f441dbda1143461876f35 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:47:41.412348 6773 raft_consensus.cc:2258] T fe72ad4a750742b0a11ac3931c5eb63d P f103722ba11f441dbda1143461876f35 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:47:41.413077 6773 ts_tablet_manager.cc:1905] T fe72ad4a750742b0a11ac3931c5eb63d P f103722ba11f441dbda1143461876f35: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:47:41.413079 6772 ts_tablet_manager.cc:1905] T fe72ad4a750742b0a11ac3931c5eb63d P 970e2afa7c324e03aa24b28063eae728: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:47:41.415089 6773 ts_tablet_manager.cc:1918] T fe72ad4a750742b0a11ac3931c5eb63d P f103722ba11f441dbda1143461876f35: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:47:41.415196 6773 log.cc:1196] T fe72ad4a750742b0a11ac3931c5eb63d P f103722ba11f441dbda1143461876f35: Deleting WAL directory at /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-1/wal/wals/fe72ad4a750742b0a11ac3931c5eb63d
I20260806 22:47:41.415458 6772 ts_tablet_manager.cc:1918] T fe72ad4a750742b0a11ac3931c5eb63d P 970e2afa7c324e03aa24b28063eae728: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:47:41.415549 6772 log.cc:1196] T fe72ad4a750742b0a11ac3931c5eb63d P 970e2afa7c324e03aa24b28063eae728: Deleting WAL directory at /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-2/wal/wals/fe72ad4a750742b0a11ac3931c5eb63d
I20260806 22:47:41.415549 6773 ts_tablet_manager.cc:1939] T fe72ad4a750742b0a11ac3931c5eb63d P f103722ba11f441dbda1143461876f35: Deleting consensus metadata
I20260806 22:47:41.415849 6772 ts_tablet_manager.cc:1939] T fe72ad4a750742b0a11ac3931c5eb63d P 970e2afa7c324e03aa24b28063eae728: Deleting consensus metadata
I20260806 22:47:41.416177 6088 catalog_manager.cc:4841] TS f103722ba11f441dbda1143461876f35 (127.18.45.194:33871): tablet fe72ad4a750742b0a11ac3931c5eb63d (table test-table [id=5f7d3059196f47f8b51a70ebd4d29d59]) successfully deleted
I20260806 22:47:41.416430 6089 catalog_manager.cc:4841] TS 970e2afa7c324e03aa24b28063eae728 (127.18.45.195:36453): tablet fe72ad4a750742b0a11ac3931c5eb63d (table test-table [id=5f7d3059196f47f8b51a70ebd4d29d59]) successfully deleted
W20260806 22:47:41.420063 6088 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.45.193:39517 (error 108)
I20260806 22:47:41.420748 6737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:41.420570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.193:39517 (local address 127.0.0.1:59508)
0806 22:47:41.420650 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:41.420701 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.193:39517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260806 22:47:41.420825 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 6365
W20260806 22:47:41.420931 6088 catalog_manager.cc:4584] TS 3c4699a06f354c3db12e2b056bd7ccfb (127.18.45.193:39517): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fe72ad4a750742b0a11ac3931c5eb63d: Network error: Client connection negotiation failed: client connection to 127.18.45.193:39517: connect: Connection refused (error 111)
I20260806 22:47:41.440073 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 6554
I20260806 22:47:41.451224 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 6062
I20260806 22:47:41.463415 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:39797
--webserver_interface=127.18.45.254
--webserver_port=35941
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:37755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.45.254:39797
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:47:41.559422 6776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:41.559674 6776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:41.559696 6776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:41.560928 6776 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:47:41.560973 6776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:41.560988 6776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:41.561002 6776 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:47:41.561013 6776 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:47:41.562610 6776 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:37755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.18.45.254:39797
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:39797
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.45.254
--webserver_port=35941
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6776
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:47:41.562911 6776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:41.563230 6776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:41.574280 6787 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:41.574282 6791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:41.574306 6788 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:41.574501 6776 server_base.cc:1034] running on GCE node
I20260806 22:47:41.574808 6776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:41.575227 6776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:41.576399 6776 hybrid_clock.cc:648] HybridClock initialized: now 1786056461576379 us; error 42 us; skew 500 ppm
Aug 06 22:47:41 dist-test-slave-h3ft krb5kdc[6039](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056461, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:41.579471 6776 init.cc:377] Logged in from keytab as kudu/127.18.45.254@KRBTEST.COM (short username kudu)
I20260806 22:47:41.580049 6776 webserver.cc:460] Webserver started at http://127.18.45.254:35941/ using document root <none> and password file <none>
I20260806 22:47:41.580340 6776 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:41.580428 6776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:41.582046 6776 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:47:41.582804 6799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:41.583068 6776 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:47:41.583194 6776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/master-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/master-0/wal
uuid: "a6658b79b3ec4a41ab783c6eb6a8f1cc"
format_stamp: "Formatted at 2026-08-06 22:47:40 on dist-test-slave-h3ft"
server_key: "10442fe2d6f5b61f368ac8bed514eec1"
server_key_iv: "2bb5c76b1bbe99b2b3eba2b2052f2fa0"
server_key_version: "encryptionkey@0"
I20260806 22:47:41.583679 6776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:41.596854 6776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:41.599238 6776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:41.599432 6776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:41.604787 6776 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.254:39797
I20260806 22:47:41.604826 6870 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.254:39797 every 8 connection(s)
I20260806 22:47:41.605327 6776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
I20260806 22:47:41.607554 6871 sys_catalog.cc:263] Verifying existing consensus state
I20260806 22:47:41.608345 6871 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6658b79b3ec4a41ab783c6eb6a8f1cc: Bootstrap starting.
I20260806 22:47:41.610483 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 6776
I20260806 22:47:41.610949 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.193:39517
--local_ip_for_outbound_sockets=127.18.45.193
--tserver_master_addrs=127.18.45.254:39797
--webserver_port=44663
--webserver_interface=127.18.45.193
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab
--principal=kudu/127.18.45.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:37755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:47:41.620997 6871 log.cc:824] T 00000000000000000000000000000000 P a6658b79b3ec4a41ab783c6eb6a8f1cc: Log is configured to *not* fsync() on all Append() calls
I20260806 22:47:41.623515 6871 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6658b79b3ec4a41ab783c6eb6a8f1cc: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:47:41.623909 6871 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6658b79b3ec4a41ab783c6eb6a8f1cc: Bootstrap complete.
I20260806 22:47:41.626722 6871 raft_consensus.cc:348] T 00000000000000000000000000000000 P a6658b79b3ec4a41ab783c6eb6a8f1cc [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6658b79b3ec4a41ab783c6eb6a8f1cc" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39797 } }
I20260806 22:47:41.627135 6871 raft_consensus.cc:729] T 00000000000000000000000000000000 P a6658b79b3ec4a41ab783c6eb6a8f1cc [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6658b79b3ec4a41ab783c6eb6a8f1cc, State: Initialized, Role: FOLLOWER
I20260806 22:47:41.627360 6871 consensus_queue.cc:260] T 00000000000000000000000000000000 P a6658b79b3ec4a41ab783c6eb6a8f1cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6658b79b3ec4a41ab783c6eb6a8f1cc" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39797 } }
I20260806 22:47:41.627437 6871 raft_consensus.cc:388] T 00000000000000000000000000000000 P a6658b79b3ec4a41ab783c6eb6a8f1cc [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:47:41.627512 6871 raft_consensus.cc:482] T 00000000000000000000000000000000 P a6658b79b3ec4a41ab783c6eb6a8f1cc [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:47:41.627568 6871 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a6658b79b3ec4a41ab783c6eb6a8f1cc [term 1 FOLLOWER]: Advancing to term 2
I20260806 22:47:41.628829 6871 raft_consensus.cc:504] T 00000000000000000000000000000000 P a6658b79b3ec4a41ab783c6eb6a8f1cc [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6658b79b3ec4a41ab783c6eb6a8f1cc" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39797 } }
I20260806 22:47:41.628929 6871 leader_election.cc:302] T 00000000000000000000000000000000 P a6658b79b3ec4a41ab783c6eb6a8f1cc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a6658b79b3ec4a41ab783c6eb6a8f1cc; no voters:
I20260806 22:47:41.629208 6871 leader_election.cc:288] T 00000000000000000000000000000000 P a6658b79b3ec4a41ab783c6eb6a8f1cc [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 22:47:41.629251 6876 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a6658b79b3ec4a41ab783c6eb6a8f1cc [term 2 FOLLOWER]: Leader election won for term 2
I20260806 22:47:41.629384 6876 raft_consensus.cc:686] T 00000000000000000000000000000000 P a6658b79b3ec4a41ab783c6eb6a8f1cc [term 2 LEADER]: Becoming Leader. State: Replica: a6658b79b3ec4a41ab783c6eb6a8f1cc, State: Running, Role: LEADER
I20260806 22:47:41.629482 6876 consensus_queue.cc:237] T 00000000000000000000000000000000 P a6658b79b3ec4a41ab783c6eb6a8f1cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6658b79b3ec4a41ab783c6eb6a8f1cc" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39797 } }
I20260806 22:47:41.629554 6871 sys_catalog.cc:564] T 00000000000000000000000000000000 P a6658b79b3ec4a41ab783c6eb6a8f1cc [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:47:41.629869 6879 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6658b79b3ec4a41ab783c6eb6a8f1cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "a6658b79b3ec4a41ab783c6eb6a8f1cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6658b79b3ec4a41ab783c6eb6a8f1cc" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39797 } } }
I20260806 22:47:41.630152 6879 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6658b79b3ec4a41ab783c6eb6a8f1cc [sys.catalog]: This master's current role is: LEADER
I20260806 22:47:41.630049 6878 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6658b79b3ec4a41ab783c6eb6a8f1cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader a6658b79b3ec4a41ab783c6eb6a8f1cc. Latest consensus state: current_term: 2 leader_uuid: "a6658b79b3ec4a41ab783c6eb6a8f1cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6658b79b3ec4a41ab783c6eb6a8f1cc" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39797 } } }
I20260806 22:47:41.630461 6878 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6658b79b3ec4a41ab783c6eb6a8f1cc [sys.catalog]: This master's current role is: LEADER
I20260806 22:47:41.630628 6887 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:47:41.631718 6887 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:47:41.631846 6887 catalog_manager.cc:1259] Loaded cluster ID: 821df2c33fa14fca8c4df6091d48ed3e
I20260806 22:47:41.631882 6887 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:47:41.632467 6887 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260806 22:47:41.637148 6887 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260806 22:47:41.640321 6887 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:47:41.640605 6887 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a6658b79b3ec4a41ab783c6eb6a8f1cc: Loaded TSK: 0
I20260806 22:47:41.640868 6887 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 22:47:41.712638 6874 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:41.712965 6874 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:41.713014 6874 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:41.714334 6874 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:41.714413 6874 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:41.714461 6874 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.193
I20260806 22:47:41.716298 6874 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:37755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab
--principal=kudu/127.18.45.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.193:39517
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.45.193
--webserver_port=44663
--tserver_master_addrs=127.18.45.254:39797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6874
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.193
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:47:41.716670 6874 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:41.716951 6874 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:41.724078 6905 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:41.724128 6907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:41.724260 6874 server_base.cc:1034] running on GCE node
W20260806 22:47:41.724027 6903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:41.724560 6874 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:41.724896 6874 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:41.726118 6874 hybrid_clock.cc:648] HybridClock initialized: now 1786056461726092 us; error 36 us; skew 500 ppm
Aug 06 22:47:41 dist-test-slave-h3ft krb5kdc[6039](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056461, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:41.729465 6874 init.cc:377] Logged in from keytab as kudu/127.18.45.193@KRBTEST.COM (short username kudu)
I20260806 22:47:41.730115 6874 webserver.cc:460] Webserver started at http://127.18.45.193:44663/ using document root <none> and password file <none>
I20260806 22:47:41.730505 6874 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:41.730612 6874 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:41.732431 6874 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:47:41.733389 6917 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:41.733606 6874 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:47:41.733708 6874 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-0/wal
uuid: "3c4699a06f354c3db12e2b056bd7ccfb"
format_stamp: "Formatted at 2026-08-06 22:47:40 on dist-test-slave-h3ft"
server_key: "ab901a5d6a2afc33687d31cb5e9d5235"
server_key_iv: "61bc0e5d62033c27349220873c9abc5d"
server_key_version: "encryptionkey@0"
I20260806 22:47:41.734306 6874 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:41.753754 6874 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:41.757171 6874 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:41.757372 6874 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:41.757709 6874 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:47:41.758392 6926 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 22:47:41.760411 6874 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 22:47:41.760480 6874 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260806 22:47:41.760524 6874 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 22:47:41.761603 6874 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 22:47:41.761668 6874 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260806 22:47:41.761695 6926 tablet_bootstrap.cc:492] T fe72ad4a750742b0a11ac3931c5eb63d P 3c4699a06f354c3db12e2b056bd7ccfb: Bootstrap starting.
I20260806 22:47:41.770639 6874 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.193:39517
I20260806 22:47:41.771226 6874 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
I20260806 22:47:41.771178 7077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.193:39517 every 8 connection(s)
I20260806 22:47:41.776232 6926 log.cc:824] T fe72ad4a750742b0a11ac3931c5eb63d P 3c4699a06f354c3db12e2b056bd7ccfb: Log is configured to *not* fsync() on all Append() calls
I20260806 22:47:41.779323 6926 tablet_bootstrap.cc:492] T fe72ad4a750742b0a11ac3931c5eb63d P 3c4699a06f354c3db12e2b056bd7ccfb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:47:41.779704 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 6874
I20260806 22:47:41.779743 6926 tablet_bootstrap.cc:492] T fe72ad4a750742b0a11ac3931c5eb63d P 3c4699a06f354c3db12e2b056bd7ccfb: Bootstrap complete.
I20260806 22:47:41.780035 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.194:33871
--local_ip_for_outbound_sockets=127.18.45.194
--tserver_master_addrs=127.18.45.254:39797
--webserver_port=39845
--webserver_interface=127.18.45.194
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab
--principal=kudu/127.18.45.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:37755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:47:41.781307 6926 ts_tablet_manager.cc:1397] T fe72ad4a750742b0a11ac3931c5eb63d P 3c4699a06f354c3db12e2b056bd7ccfb: Time spent bootstrapping tablet: real 0.020s user 0.012s sys 0.005s
I20260806 22:47:41.783239 6926 raft_consensus.cc:348] T fe72ad4a750742b0a11ac3931c5eb63d P 3c4699a06f354c3db12e2b056bd7ccfb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "970e2afa7c324e03aa24b28063eae728" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 36453 } } peers { permanent_uuid: "f103722ba11f441dbda1143461876f35" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33871 } } peers { permanent_uuid: "3c4699a06f354c3db12e2b056bd7ccfb" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 39517 } }
I20260806 22:47:41.783618 6926 raft_consensus.cc:729] T fe72ad4a750742b0a11ac3931c5eb63d P 3c4699a06f354c3db12e2b056bd7ccfb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c4699a06f354c3db12e2b056bd7ccfb, State: Initialized, Role: FOLLOWER
I20260806 22:47:41.783818 6926 consensus_queue.cc:260] T fe72ad4a750742b0a11ac3931c5eb63d P 3c4699a06f354c3db12e2b056bd7ccfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "970e2afa7c324e03aa24b28063eae728" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 36453 } } peers { permanent_uuid: "f103722ba11f441dbda1143461876f35" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33871 } } peers { permanent_uuid: "3c4699a06f354c3db12e2b056bd7ccfb" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 39517 } }
I20260806 22:47:41.784200 6926 ts_tablet_manager.cc:1428] T fe72ad4a750742b0a11ac3931c5eb63d P 3c4699a06f354c3db12e2b056bd7ccfb: Time spent starting tablet: real 0.003s user 0.005s sys 0.000s
Aug 06 22:47:41 dist-test-slave-h3ft krb5kdc[6039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056461, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.193@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:47:41.793359 7080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:41.772282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39797 (local address 127.18.45.193:46065)
0806 22:47:41.775201 (+ 2919us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:41.775277 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:41.776072 (+ 795us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:41.776785 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:41.776789 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:41.777255 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:41.777561 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:41.777569 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:41.779398 (+ 1829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:41.779399 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:41.783782 (+ 4383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:41.783786 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:41.784753 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:41.784754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:41.784868 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:41.785646 (+ 778us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:41.785659 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:41.787941 (+ 2282us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:41.791062 (+ 3121us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:41.791074 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:41.791077 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:41.791370 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:41.791761 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:41.791762 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:41.791762 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:41.791950 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:41.792711 (+ 761us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:41.792719 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:41.792871 (+ 152us) client_negotiation.cc:769] Sending connection context
0806 22:47:41.792972 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 22:47:41.793150 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2844,"thread_start_us":107,"threads_started":1}
I20260806 22:47:41.793565 7081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:41.772417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:46065 (local address 127.18.45.254:39797)
0806 22:47:41.772757 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:41.772770 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:41.775474 (+ 2704us) server_negotiation.cc:407] Connection header received
0806 22:47:41.776371 (+ 897us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:41.776379 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:41.776458 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:41.776643 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:41.777709 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:41.779249 (+ 1540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:41.783919 (+ 4670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:41.784555 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:41.784632 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:41.788120 (+ 3488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:41.788127 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:41.788128 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:41.788148 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:41.790884 (+ 2736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:41.791498 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:41.791503 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:41.791504 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:41.791679 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:41.792146 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:41.792148 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:41.792148 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:41.792626 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:41.792741 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:41.793088 (+ 347us) server_negotiation.cc:299] Negotiation successful
0806 22:47:41.793285 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":109,"threads_started":1}
I20260806 22:47:41.794418 7078 heartbeater.cc:347] Connected to a master server at 127.18.45.254:39797
I20260806 22:47:41.794574 7078 heartbeater.cc:464] Registering TS with master...
I20260806 22:47:41.794872 7078 heartbeater.cc:511] Master 127.18.45.254:39797 requested a full tablet report, sending...
I20260806 22:47:41.795902 6821 ts_manager.cc:194] Registered new tserver with Master: 3c4699a06f354c3db12e2b056bd7ccfb (127.18.45.193:39517)
I20260806 22:47:41.797256 6821 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.45.193@KRBTEST.COM'} at 127.18.45.193:46065
I20260806 22:47:41.798463 7078 heartbeater.cc:503] Master 127.18.45.254:39797 was elected leader, sending a full tablet report...
Aug 06 22:47:41 dist-test-slave-h3ft krb5kdc[6039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056461, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.254@KRBTEST.COM for kudu/127.18.45.193@KRBTEST.COM
I20260806 22:47:41.814458 7086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:41.796757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.193:39517 (local address 127.0.0.1:59522)
0806 22:47:41.797137 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:41.797180 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:41.797278 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:41.797676 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:41.797706 (+ 30us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:41.798099 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:41.798174 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:41.798178 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:41.799561 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:41.799562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:41.800565 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:41.800568 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:41.801308 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:41.801309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:41.801398 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:41.801964 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:41.801971 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:41.804201 (+ 2230us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:41.806522 (+ 2321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:41.806531 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:41.806534 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:41.806815 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:41.813205 (+ 6390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:41.813207 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:41.813207 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:41.813351 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:41.813919 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:41.813936 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:41.814015 (+ 79us) client_negotiation.cc:769] Sending connection context
0806 22:47:41.814107 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 22:47:41.814324 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":151,"threads_started":1}
I20260806 22:47:41.815019 7087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:41.796849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59522 (local address 127.18.45.193:39517)
0806 22:47:41.797059 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:41.797064 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:41.797277 (+ 213us) server_negotiation.cc:407] Connection header received
0806 22:47:41.797481 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:41.797483 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:41.797511 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:41.797592 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:41.798290 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:41.799440 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:41.800668 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:41.801162 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:41.801230 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:41.804327 (+ 3097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:41.804341 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:41.804341 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:41.804351 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:41.806410 (+ 2059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:41.807076 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:41.807080 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:41.807081 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:41.807179 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:41.813475 (+ 6296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:41.813477 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:41.813478 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:41.813789 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:41.813883 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:41.814743 (+ 860us) server_negotiation.cc:299] Negotiation successful
0806 22:47:41.814891 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":113,"threads_started":1}
I20260806 22:47:41.815732 6982 tablet_service.cc:1514] Processing DeleteTablet for tablet fe72ad4a750742b0a11ac3931c5eb63d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:47:41 UTC) from {username='kudu', principal='kudu/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:59522
I20260806 22:47:41.816514 7088 tablet_replica.cc:330] stopping tablet replica
I20260806 22:47:41.816669 7088 raft_consensus.cc:2229] T fe72ad4a750742b0a11ac3931c5eb63d P 3c4699a06f354c3db12e2b056bd7ccfb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:47:41.816772 7088 raft_consensus.cc:2258] T fe72ad4a750742b0a11ac3931c5eb63d P 3c4699a06f354c3db12e2b056bd7ccfb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:47:41.817175 7088 ts_tablet_manager.cc:1905] T fe72ad4a750742b0a11ac3931c5eb63d P 3c4699a06f354c3db12e2b056bd7ccfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:47:41.819558 7088 ts_tablet_manager.cc:1918] T fe72ad4a750742b0a11ac3931c5eb63d P 3c4699a06f354c3db12e2b056bd7ccfb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:47:41.819664 7088 log.cc:1196] T fe72ad4a750742b0a11ac3931c5eb63d P 3c4699a06f354c3db12e2b056bd7ccfb: Deleting WAL directory at /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-0/wal/wals/fe72ad4a750742b0a11ac3931c5eb63d
I20260806 22:47:41.819962 7088 ts_tablet_manager.cc:1939] T fe72ad4a750742b0a11ac3931c5eb63d P 3c4699a06f354c3db12e2b056bd7ccfb: Deleting consensus metadata
I20260806 22:47:41.821082 6802 catalog_manager.cc:4841] TS 3c4699a06f354c3db12e2b056bd7ccfb (127.18.45.193:39517): tablet fe72ad4a750742b0a11ac3931c5eb63d (table test-table [id=5f7d3059196f47f8b51a70ebd4d29d59]) successfully deleted
W20260806 22:47:41.902776 7084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:41.903128 7084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:41.903165 7084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:41.904448 7084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:41.904495 7084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:41.904523 7084 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.194
I20260806 22:47:41.906118 7084 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:37755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab
--principal=kudu/127.18.45.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.194:33871
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.45.194
--webserver_port=39845
--tserver_master_addrs=127.18.45.254:39797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7084
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.194
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:47:41.906414 7084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:41.906714 7084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:41.912919 7098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:41.912963 7095 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:41.913009 7096 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:41.913313 7084 server_base.cc:1034] running on GCE node
I20260806 22:47:41.913558 7084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:41.913803 7084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:41.914973 7084 hybrid_clock.cc:648] HybridClock initialized: now 1786056461914948 us; error 40 us; skew 500 ppm
Aug 06 22:47:41 dist-test-slave-h3ft krb5kdc[6039](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056461, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:41.918071 7084 init.cc:377] Logged in from keytab as kudu/127.18.45.194@KRBTEST.COM (short username kudu)
I20260806 22:47:41.918596 7084 webserver.cc:460] Webserver started at http://127.18.45.194:39845/ using document root <none> and password file <none>
I20260806 22:47:41.918875 7084 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:41.918975 7084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:41.920583 7084 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:47:41.921283 7109 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:41.921537 7084 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:47:41.921627 7084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-1/wal
uuid: "f103722ba11f441dbda1143461876f35"
format_stamp: "Formatted at 2026-08-06 22:47:40 on dist-test-slave-h3ft"
server_key: "92f81981150686adb8d499e315f3f9bb"
server_key_iv: "1b051bfa9ea92fc75a91a60048c980ac"
server_key_version: "encryptionkey@0"
I20260806 22:47:41.922078 7084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:41.934799 7084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:41.937322 7084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:41.937520 7084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:41.937832 7084 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:47:41.938215 7084 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:47:41.938249 7084 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:41.938277 7084 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:47:41.938321 7084 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:41.946112 7084 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.194:33871
I20260806 22:47:41.946159 7259 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.194:33871 every 8 connection(s)
I20260806 22:47:41.946584 7084 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
I20260806 22:47:41.950462 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 7084
I20260806 22:47:41.950781 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.195:36453
--local_ip_for_outbound_sockets=127.18.45.195
--tserver_master_addrs=127.18.45.254:39797
--webserver_port=44609
--webserver_interface=127.18.45.195
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab
--principal=kudu/127.18.45.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:37755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 22:47:41 dist-test-slave-h3ft krb5kdc[6039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056461, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.194@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:47:41.958634 7081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:41.947540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:45501 (local address 127.18.45.254:39797)
0806 22:47:41.947602 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:41.947604 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:41.947854 (+ 250us) server_negotiation.cc:407] Connection header received
0806 22:47:41.948603 (+ 749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:41.948605 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:41.948621 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:41.948689 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:41.949864 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:41.950439 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:41.951684 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:41.951920 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:41.952056 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:41.954908 (+ 2852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:41.954909 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:41.954910 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:41.954918 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:41.956646 (+ 1728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:41.957296 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:41.957298 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:41.957298 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:41.957378 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:41.957768 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:41.957770 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:41.957770 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:41.957915 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:41.957991 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:41.958435 (+ 444us) server_negotiation.cc:299] Negotiation successful
0806 22:47:41.958548 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 22:47:41.958989 7264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:41.947514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39797 (local address 127.18.45.194:45501)
0806 22:47:41.947704 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:41.947748 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:41.948471 (+ 723us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:41.948774 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:41.948778 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:41.949511 (+ 733us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:41.949752 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:41.949756 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:41.950549 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:41.950550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:41.951585 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:41.951588 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:41.952072 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:41.952074 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:41.952217 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:41.952872 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:41.952882 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:41.954765 (+ 1883us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:41.956743 (+ 1978us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:41.956753 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:41.956757 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:41.957184 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:41.957466 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:41.957466 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:41.957467 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:41.957651 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:41.957998 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:41.958008 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:41.958242 (+ 234us) client_negotiation.cc:769] Sending connection context
0806 22:47:41.958392 (+ 150us) client_negotiation.cc:240] Negotiation successful
0806 22:47:41.958642 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":84,"threads_started":1}
I20260806 22:47:41.959601 7261 heartbeater.cc:347] Connected to a master server at 127.18.45.254:39797
I20260806 22:47:41.959723 7261 heartbeater.cc:464] Registering TS with master...
I20260806 22:47:41.959976 7261 heartbeater.cc:511] Master 127.18.45.254:39797 requested a full tablet report, sending...
I20260806 22:47:41.960480 6821 ts_manager.cc:194] Registered new tserver with Master: f103722ba11f441dbda1143461876f35 (127.18.45.194:33871)
I20260806 22:47:41.960908 6821 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.45.194@KRBTEST.COM'} at 127.18.45.194:45501
W20260806 22:47:42.048679 7265 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:42.048974 7265 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:42.049012 7265 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:42.050210 7265 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:42.050282 7265 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:42.050324 7265 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.195
I20260806 22:47:42.051959 7265 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:37755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab
--principal=kudu/127.18.45.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.195:36453
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.45.195
--webserver_port=44609
--tserver_master_addrs=127.18.45.254:39797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7265
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.195
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:47:42.052258 7265 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:42.052526 7265 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:42.058635 7275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:42.058661 7272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:42.058657 7273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:42.059163 7265 server_base.cc:1034] running on GCE node
I20260806 22:47:42.059404 7265 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:42.059676 7265 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:42.060851 7265 hybrid_clock.cc:648] HybridClock initialized: now 1786056462060826 us; error 43 us; skew 500 ppm
Aug 06 22:47:42 dist-test-slave-h3ft krb5kdc[6039](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056462, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:42.063830 7265 init.cc:377] Logged in from keytab as kudu/127.18.45.195@KRBTEST.COM (short username kudu)
I20260806 22:47:42.064384 7265 webserver.cc:460] Webserver started at http://127.18.45.195:44609/ using document root <none> and password file <none>
I20260806 22:47:42.064641 7265 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:42.064741 7265 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:42.066269 7265 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:47:42.067075 7282 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:42.067263 7265 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:47:42.067377 7265 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-2/wal
uuid: "970e2afa7c324e03aa24b28063eae728"
format_stamp: "Formatted at 2026-08-06 22:47:40 on dist-test-slave-h3ft"
server_key: "1b0acecf3389179e2102ca862d630c30"
server_key_iv: "6cb2d268c209228a8f4b607284fa7c66"
server_key_version: "encryptionkey@0"
I20260806 22:47:42.067821 7265 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:42.096396 7265 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:42.099300 7265 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:42.099526 7265 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:42.099869 7265 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:47:42.100299 7265 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:47:42.100363 7265 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:42.100420 7265 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:47:42.100452 7265 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:42.108139 7265 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.195:36453
I20260806 22:47:42.108208 7430 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.195:36453 every 8 connection(s)
I20260806 22:47:42.108697 7265 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
I20260806 22:47:42.109421 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 7265
Aug 06 22:47:42 dist-test-slave-h3ft krb5kdc[6039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056462, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.195@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:47:42.118669 7436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:42.110107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35668 (local address 127.18.45.254:39797)
0806 22:47:42.110234 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:42.110236 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:42.110246 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:47:42.110333 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:42.110334 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:42.110348 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:42.110457 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:42.111369 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:42.111960 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:42.112840 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:42.113076 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:42.113152 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:42.114881 (+ 1729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:42.114882 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:42.114883 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:42.114891 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:42.116733 (+ 1842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:42.117105 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:42.117107 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:42.117108 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:42.117183 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:42.117787 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:42.117789 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:42.117789 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:42.117955 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:42.118205 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:42.118435 (+ 230us) server_negotiation.cc:299] Negotiation successful
0806 22:47:42.118525 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":68,"threads_started":1}
I20260806 22:47:42.120369 7081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:42.109793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.195:37521 (local address 127.18.45.254:39797)
0806 22:47:42.109876 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:42.109879 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:42.110121 (+ 242us) server_negotiation.cc:407] Connection header received
0806 22:47:42.110807 (+ 686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:42.110808 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:42.110821 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:42.110886 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:42.112050 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:42.112583 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:42.113904 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:42.114210 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:42.114308 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:42.117244 (+ 2936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:42.117245 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:42.117245 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:42.117252 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:42.118862 (+ 1610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:42.119363 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:42.119365 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:42.119365 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:42.119424 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:42.119726 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:42.119727 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:42.119727 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:42.119876 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:42.119942 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:42.120240 (+ 298us) server_negotiation.cc:299] Negotiation successful
0806 22:47:42.120305 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 22:47:42.120646 7433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:42.109771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39797 (local address 127.18.45.195:37521)
0806 22:47:42.109962 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:42.110014 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:42.110697 (+ 683us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:42.110967 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:42.110971 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:42.111683 (+ 712us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:42.111938 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:42.111945 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:42.112698 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:42.112699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:42.113780 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:42.113784 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:42.114351 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:42.114353 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:42.114496 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:42.115113 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:42.115122 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:42.117144 (+ 2022us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:42.118935 (+ 1791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:42.118944 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:42.118947 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:42.119302 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:42.119518 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:42.119519 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:42.119519 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:42.119649 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:42.119951 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:42.119957 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:42.120103 (+ 146us) client_negotiation.cc:769] Sending connection context
0806 22:47:42.120198 (+ 95us) client_negotiation.cc:240] Negotiation successful
0806 22:47:42.120367 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":102,"threads_started":1}
I20260806 22:47:42.121343 7431 heartbeater.cc:347] Connected to a master server at 127.18.45.254:39797
I20260806 22:47:42.121491 7431 heartbeater.cc:464] Registering TS with master...
I20260806 22:47:42.121733 7431 heartbeater.cc:511] Master 127.18.45.254:39797 requested a full tablet report, sending...
I20260806 22:47:42.122282 6821 ts_manager.cc:194] Registered new tserver with Master: 970e2afa7c324e03aa24b28063eae728 (127.18.45.195:36453)
I20260806 22:47:42.122756 6821 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.45.195@KRBTEST.COM'} at 127.18.45.195:37521
I20260806 22:47:42.131201 18615 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:47:42.134670 7081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:42.131691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35674 (local address 127.18.45.254:39797)
0806 22:47:42.131773 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:42.131776 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:42.131898 (+ 122us) server_negotiation.cc:407] Connection header received
0806 22:47:42.132002 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:42.132003 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:42.132079 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:42.132192 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:47:42.132359 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:42.132912 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:42.133773 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:42.133998 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:42.134077 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:42.134271 (+ 194us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:47:42.134412 (+ 141us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:47:42.134523 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:42.134541 (+ 18us) server_negotiation.cc:299] Negotiation successful
0806 22:47:42.134570 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 22:47:42.135100 6821 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:35674:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:47:42.137181 6821 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:47:42.140278 6982 tablet_service.cc:1467] Processing CreateTablet for tablet 2c62a5ffa9e24703bcb0568cd43588f8 (DEFAULT_TABLE table=test-table [id=ca9f3f42d9424c94ab2be7d8993609be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:47:42.140569 6982 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c62a5ffa9e24703bcb0568cd43588f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:47:42.142789 6926 tablet_bootstrap.cc:492] T 2c62a5ffa9e24703bcb0568cd43588f8 P 3c4699a06f354c3db12e2b056bd7ccfb: Bootstrap starting.
I20260806 22:47:42.143533 6926 tablet_bootstrap.cc:654] T 2c62a5ffa9e24703bcb0568cd43588f8 P 3c4699a06f354c3db12e2b056bd7ccfb: Neither blocks nor log segments found. Creating new log.
I20260806 22:47:42.144943 7441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:42.140031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43504 (local address 127.18.45.195:36453)
0806 22:47:42.140319 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:42.140325 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:42.140663 (+ 338us) server_negotiation.cc:407] Connection header received
0806 22:47:42.140750 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:42.140753 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:42.140771 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:42.140856 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:47:42.141100 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:42.141940 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:42.143629 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:42.144420 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:42.144476 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:42.144552 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:42.144787 (+ 235us) server_negotiation.cc:299] Negotiation successful
0806 22:47:42.144842 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":179,"threads_started":1}
I20260806 22:47:42.145002 7086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:42.139779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.194:33871 (local address 127.0.0.1:54344)
0806 22:47:42.139927 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:42.139940 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:42.139979 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:42.140902 (+ 923us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:42.140904 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:42.140906 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:47:42.140973 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:42.140975 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:42.142517 (+ 1542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:42.142518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:42.143571 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:42.143573 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:42.144512 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:42.144513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:42.144651 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:42.144681 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 22:47:42.144772 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 22:47:42.144893 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":10}
I20260806 22:47:42.145426 7443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:42.140347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54344 (local address 127.18.45.194:33871)
0806 22:47:42.140560 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:42.140565 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:42.140578 (+ 13us) server_negotiation.cc:407] Connection header received
0806 22:47:42.140728 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:42.140731 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:42.140756 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:42.141048 (+ 292us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:47:42.141316 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:42.142391 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:42.143665 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:42.144422 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:42.144508 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:42.144561 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:42.145171 (+ 610us) server_negotiation.cc:299] Negotiation successful
0806 22:47:42.145255 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":112,"threads_started":1}
I20260806 22:47:42.145226 7440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:42.139822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.195:36453 (local address 127.0.0.1:43504)
0806 22:47:42.140579 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:42.140592 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:42.140668 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:42.140901 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:42.140902 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:42.140903 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:47:42.141004 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:42.141006 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:42.142064 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:42.142065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:42.143529 (+ 1464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:42.143531 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:42.144625 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:42.144626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:42.144695 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:42.144699 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:47:42.145098 (+ 399us) client_negotiation.cc:240] Negotiation successful
0806 22:47:42.145156 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":739,"thread_start_us":86,"threads_started":1}
I20260806 22:47:42.146339 7341 tablet_service.cc:1467] Processing CreateTablet for tablet 2c62a5ffa9e24703bcb0568cd43588f8 (DEFAULT_TABLE table=test-table [id=ca9f3f42d9424c94ab2be7d8993609be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:47:42.146353 7169 tablet_service.cc:1467] Processing CreateTablet for tablet 2c62a5ffa9e24703bcb0568cd43588f8 (DEFAULT_TABLE table=test-table [id=ca9f3f42d9424c94ab2be7d8993609be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:47:42.146776 7169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c62a5ffa9e24703bcb0568cd43588f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:47:42.146776 7341 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c62a5ffa9e24703bcb0568cd43588f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:47:42.151852 6926 tablet_bootstrap.cc:492] T 2c62a5ffa9e24703bcb0568cd43588f8 P 3c4699a06f354c3db12e2b056bd7ccfb: No bootstrap required, opened a new log
I20260806 22:47:42.151935 6926 ts_tablet_manager.cc:1397] T 2c62a5ffa9e24703bcb0568cd43588f8 P 3c4699a06f354c3db12e2b056bd7ccfb: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20260806 22:47:42.152362 6926 raft_consensus.cc:348] T 2c62a5ffa9e24703bcb0568cd43588f8 P 3c4699a06f354c3db12e2b056bd7ccfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f103722ba11f441dbda1143461876f35" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33871 } } peers { permanent_uuid: "970e2afa7c324e03aa24b28063eae728" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 36453 } } peers { permanent_uuid: "3c4699a06f354c3db12e2b056bd7ccfb" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 39517 } }
I20260806 22:47:42.152474 6926 raft_consensus.cc:374] T 2c62a5ffa9e24703bcb0568cd43588f8 P 3c4699a06f354c3db12e2b056bd7ccfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:47:42.152535 6926 raft_consensus.cc:729] T 2c62a5ffa9e24703bcb0568cd43588f8 P 3c4699a06f354c3db12e2b056bd7ccfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c4699a06f354c3db12e2b056bd7ccfb, State: Initialized, Role: FOLLOWER
I20260806 22:47:42.152623 6926 consensus_queue.cc:260] T 2c62a5ffa9e24703bcb0568cd43588f8 P 3c4699a06f354c3db12e2b056bd7ccfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f103722ba11f441dbda1143461876f35" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33871 } } peers { permanent_uuid: "970e2afa7c324e03aa24b28063eae728" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 36453 } } peers { permanent_uuid: "3c4699a06f354c3db12e2b056bd7ccfb" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 39517 } }
I20260806 22:47:42.152742 6926 ts_tablet_manager.cc:1428] T 2c62a5ffa9e24703bcb0568cd43588f8 P 3c4699a06f354c3db12e2b056bd7ccfb: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 22:47:42.153064 7446 tablet_bootstrap.cc:492] T 2c62a5ffa9e24703bcb0568cd43588f8 P f103722ba11f441dbda1143461876f35: Bootstrap starting.
I20260806 22:47:42.153781 7446 tablet_bootstrap.cc:654] T 2c62a5ffa9e24703bcb0568cd43588f8 P f103722ba11f441dbda1143461876f35: Neither blocks nor log segments found. Creating new log.
I20260806 22:47:42.154429 7446 log.cc:824] T 2c62a5ffa9e24703bcb0568cd43588f8 P f103722ba11f441dbda1143461876f35: Log is configured to *not* fsync() on all Append() calls
I20260806 22:47:42.155364 7447 tablet_bootstrap.cc:492] T 2c62a5ffa9e24703bcb0568cd43588f8 P 970e2afa7c324e03aa24b28063eae728: Bootstrap starting.
I20260806 22:47:42.155669 7446 tablet_bootstrap.cc:492] T 2c62a5ffa9e24703bcb0568cd43588f8 P f103722ba11f441dbda1143461876f35: No bootstrap required, opened a new log
I20260806 22:47:42.155747 7446 ts_tablet_manager.cc:1397] T 2c62a5ffa9e24703bcb0568cd43588f8 P f103722ba11f441dbda1143461876f35: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:47:42.156327 7447 tablet_bootstrap.cc:654] T 2c62a5ffa9e24703bcb0568cd43588f8 P 970e2afa7c324e03aa24b28063eae728: Neither blocks nor log segments found. Creating new log.
I20260806 22:47:42.157068 7447 log.cc:824] T 2c62a5ffa9e24703bcb0568cd43588f8 P 970e2afa7c324e03aa24b28063eae728: Log is configured to *not* fsync() on all Append() calls
I20260806 22:47:42.157650 7446 raft_consensus.cc:348] T 2c62a5ffa9e24703bcb0568cd43588f8 P f103722ba11f441dbda1143461876f35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f103722ba11f441dbda1143461876f35" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33871 } } peers { permanent_uuid: "970e2afa7c324e03aa24b28063eae728" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 36453 } } peers { permanent_uuid: "3c4699a06f354c3db12e2b056bd7ccfb" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 39517 } }
I20260806 22:47:42.157860 7446 raft_consensus.cc:374] T 2c62a5ffa9e24703bcb0568cd43588f8 P f103722ba11f441dbda1143461876f35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:47:42.157910 7446 raft_consensus.cc:729] T 2c62a5ffa9e24703bcb0568cd43588f8 P f103722ba11f441dbda1143461876f35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f103722ba11f441dbda1143461876f35, State: Initialized, Role: FOLLOWER
I20260806 22:47:42.158056 7446 consensus_queue.cc:260] T 2c62a5ffa9e24703bcb0568cd43588f8 P f103722ba11f441dbda1143461876f35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f103722ba11f441dbda1143461876f35" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33871 } } peers { permanent_uuid: "970e2afa7c324e03aa24b28063eae728" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 36453 } } peers { permanent_uuid: "3c4699a06f354c3db12e2b056bd7ccfb" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 39517 } }
I20260806 22:47:42.158138 7447 tablet_bootstrap.cc:492] T 2c62a5ffa9e24703bcb0568cd43588f8 P 970e2afa7c324e03aa24b28063eae728: No bootstrap required, opened a new log
I20260806 22:47:42.158202 7447 ts_tablet_manager.cc:1397] T 2c62a5ffa9e24703bcb0568cd43588f8 P 970e2afa7c324e03aa24b28063eae728: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:47:42.158435 7446 ts_tablet_manager.cc:1428] T 2c62a5ffa9e24703bcb0568cd43588f8 P f103722ba11f441dbda1143461876f35: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:47:42.158479 7261 heartbeater.cc:503] Master 127.18.45.254:39797 was elected leader, sending a full tablet report...
I20260806 22:47:42.160338 7447 raft_consensus.cc:348] T 2c62a5ffa9e24703bcb0568cd43588f8 P 970e2afa7c324e03aa24b28063eae728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f103722ba11f441dbda1143461876f35" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33871 } } peers { permanent_uuid: "970e2afa7c324e03aa24b28063eae728" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 36453 } } peers { permanent_uuid: "3c4699a06f354c3db12e2b056bd7ccfb" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 39517 } }
I20260806 22:47:42.160523 7447 raft_consensus.cc:374] T 2c62a5ffa9e24703bcb0568cd43588f8 P 970e2afa7c324e03aa24b28063eae728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:47:42.160554 7447 raft_consensus.cc:729] T 2c62a5ffa9e24703bcb0568cd43588f8 P 970e2afa7c324e03aa24b28063eae728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 970e2afa7c324e03aa24b28063eae728, State: Initialized, Role: FOLLOWER
I20260806 22:47:42.160693 7447 consensus_queue.cc:260] T 2c62a5ffa9e24703bcb0568cd43588f8 P 970e2afa7c324e03aa24b28063eae728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f103722ba11f441dbda1143461876f35" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33871 } } peers { permanent_uuid: "970e2afa7c324e03aa24b28063eae728" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 36453 } } peers { permanent_uuid: "3c4699a06f354c3db12e2b056bd7ccfb" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 39517 } }
I20260806 22:47:42.160979 7447 ts_tablet_manager.cc:1428] T 2c62a5ffa9e24703bcb0568cd43588f8 P 970e2afa7c324e03aa24b28063eae728: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:47:42.161114 7431 heartbeater.cc:503] Master 127.18.45.254:39797 was elected leader, sending a full tablet report...
I20260806 22:47:42.161815 7453 raft_consensus.cc:482] T 2c62a5ffa9e24703bcb0568cd43588f8 P 970e2afa7c324e03aa24b28063eae728 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:47:42.161901 7453 raft_consensus.cc:504] T 2c62a5ffa9e24703bcb0568cd43588f8 P 970e2afa7c324e03aa24b28063eae728 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f103722ba11f441dbda1143461876f35" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33871 } } peers { permanent_uuid: "970e2afa7c324e03aa24b28063eae728" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 36453 } } peers { permanent_uuid: "3c4699a06f354c3db12e2b056bd7ccfb" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 39517 } }
I20260806 22:47:42.162226 7453 leader_election.cc:288] T 2c62a5ffa9e24703bcb0568cd43588f8 P 970e2afa7c324e03aa24b28063eae728 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f103722ba11f441dbda1143461876f35 (127.18.45.194:33871), 3c4699a06f354c3db12e2b056bd7ccfb (127.18.45.193:39517)
I20260806 22:47:42.166112 7443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:42.162527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.195:42325 (local address 127.18.45.194:33871)
0806 22:47:42.162572 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:42.162574 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:42.162615 (+ 41us) server_negotiation.cc:407] Connection header received
0806 22:47:42.162640 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:42.162641 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:42.162652 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:42.162702 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:47:42.162914 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:42.163910 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:42.165140 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:42.165686 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:42.165784 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:42.165797 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:42.165984 (+ 187us) server_negotiation.cc:299] Negotiation successful
0806 22:47:42.166032 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 22:47:42.166112 7433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:42.162430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.194:33871 (local address 127.18.45.195:42325)
0806 22:47:42.162500 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:42.162516 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:42.162563 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:42.162707 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:42.162708 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:42.162710 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:47:42.162814 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:42.162816 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:42.163997 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:42.163999 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:42.165028 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:42.165031 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:42.165777 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:42.165778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:42.165878 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:42.165882 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:47:42.165974 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 22:47:42.166022 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260806 22:47:42.166525 7455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:42.162565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.193:39517 (local address 127.18.45.195:43359)
0806 22:47:42.162771 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:42.162781 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:42.162836 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:42.163117 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:42.163118 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:42.163119 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:47:42.163207 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:42.163209 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:42.164169 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:42.164170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:42.165351 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:42.165352 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:42.166281 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:42.166282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:42.166353 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:42.166357 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:47:42.166404 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 22:47:42.166438 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":153,"threads_started":1}
I20260806 22:47:42.166584 7087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:42.162848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.195:43359 (local address 127.18.45.193:39517)
0806 22:47:42.162906 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:42.162908 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:42.162917 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:47:42.162947 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:42.162948 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:42.162978 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:42.163065 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:47:42.163292 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:42.164075 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:42.165445 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:42.166119 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:42.166162 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:42.166222 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:42.166433 (+ 211us) server_negotiation.cc:299] Negotiation successful
0806 22:47:42.166487 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 22:47:42.166887 7011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c62a5ffa9e24703bcb0568cd43588f8" candidate_uuid: "970e2afa7c324e03aa24b28063eae728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c4699a06f354c3db12e2b056bd7ccfb" is_pre_election: true
I20260806 22:47:42.166918 7199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c62a5ffa9e24703bcb0568cd43588f8" candidate_uuid: "970e2afa7c324e03aa24b28063eae728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f103722ba11f441dbda1143461876f35" is_pre_election: true
I20260806 22:47:42.167124 7011 raft_consensus.cc:2454] T 2c62a5ffa9e24703bcb0568cd43588f8 P 3c4699a06f354c3db12e2b056bd7ccfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 970e2afa7c324e03aa24b28063eae728 in term 0.
I20260806 22:47:42.167124 7199 raft_consensus.cc:2454] T 2c62a5ffa9e24703bcb0568cd43588f8 P f103722ba11f441dbda1143461876f35 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 970e2afa7c324e03aa24b28063eae728 in term 0.
I20260806 22:47:42.167392 7285 leader_election.cc:302] T 2c62a5ffa9e24703bcb0568cd43588f8 P 970e2afa7c324e03aa24b28063eae728 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 970e2afa7c324e03aa24b28063eae728, f103722ba11f441dbda1143461876f35; no voters:
I20260806 22:47:42.167605 7453 raft_consensus.cc:2781] T 2c62a5ffa9e24703bcb0568cd43588f8 P 970e2afa7c324e03aa24b28063eae728 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:47:42.167685 7453 raft_consensus.cc:482] T 2c62a5ffa9e24703bcb0568cd43588f8 P 970e2afa7c324e03aa24b28063eae728 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:47:42.167704 7453 raft_consensus.cc:3037] T 2c62a5ffa9e24703bcb0568cd43588f8 P 970e2afa7c324e03aa24b28063eae728 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:47:42.168797 7453 raft_consensus.cc:504] T 2c62a5ffa9e24703bcb0568cd43588f8 P 970e2afa7c324e03aa24b28063eae728 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f103722ba11f441dbda1143461876f35" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33871 } } peers { permanent_uuid: "970e2afa7c324e03aa24b28063eae728" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 36453 } } peers { permanent_uuid: "3c4699a06f354c3db12e2b056bd7ccfb" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 39517 } }
I20260806 22:47:42.168939 7453 leader_election.cc:288] T 2c62a5ffa9e24703bcb0568cd43588f8 P 970e2afa7c324e03aa24b28063eae728 [CANDIDATE]: Term 1 election: Requested vote from peers f103722ba11f441dbda1143461876f35 (127.18.45.194:33871), 3c4699a06f354c3db12e2b056bd7ccfb (127.18.45.193:39517)
I20260806 22:47:42.169176 7199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c62a5ffa9e24703bcb0568cd43588f8" candidate_uuid: "970e2afa7c324e03aa24b28063eae728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f103722ba11f441dbda1143461876f35"
I20260806 22:47:42.169260 7199 raft_consensus.cc:3037] T 2c62a5ffa9e24703bcb0568cd43588f8 P f103722ba11f441dbda1143461876f35 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:47:42.170082 7199 raft_consensus.cc:2454] T 2c62a5ffa9e24703bcb0568cd43588f8 P f103722ba11f441dbda1143461876f35 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 970e2afa7c324e03aa24b28063eae728 in term 1.
I20260806 22:47:42.170249 7285 leader_election.cc:302] T 2c62a5ffa9e24703bcb0568cd43588f8 P 970e2afa7c324e03aa24b28063eae728 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 970e2afa7c324e03aa24b28063eae728, f103722ba11f441dbda1143461876f35; no voters:
I20260806 22:47:42.170223 7011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c62a5ffa9e24703bcb0568cd43588f8" candidate_uuid: "970e2afa7c324e03aa24b28063eae728" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c4699a06f354c3db12e2b056bd7ccfb"
I20260806 22:47:42.170332 7453 raft_consensus.cc:2781] T 2c62a5ffa9e24703bcb0568cd43588f8 P 970e2afa7c324e03aa24b28063eae728 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:47:42.170312 7011 raft_consensus.cc:3037] T 2c62a5ffa9e24703bcb0568cd43588f8 P 3c4699a06f354c3db12e2b056bd7ccfb [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:47:42.170511 7453 raft_consensus.cc:686] T 2c62a5ffa9e24703bcb0568cd43588f8 P 970e2afa7c324e03aa24b28063eae728 [term 1 LEADER]: Becoming Leader. State: Replica: 970e2afa7c324e03aa24b28063eae728, State: Running, Role: LEADER
I20260806 22:47:42.170617 7453 consensus_queue.cc:237] T 2c62a5ffa9e24703bcb0568cd43588f8 P 970e2afa7c324e03aa24b28063eae728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f103722ba11f441dbda1143461876f35" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33871 } } peers { permanent_uuid: "970e2afa7c324e03aa24b28063eae728" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 36453 } } peers { permanent_uuid: "3c4699a06f354c3db12e2b056bd7ccfb" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 39517 } }
I20260806 22:47:42.171464 7011 raft_consensus.cc:2454] T 2c62a5ffa9e24703bcb0568cd43588f8 P 3c4699a06f354c3db12e2b056bd7ccfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 970e2afa7c324e03aa24b28063eae728 in term 1.
I20260806 22:47:42.171643 6821 catalog_manager.cc:5495] T 2c62a5ffa9e24703bcb0568cd43588f8 P 970e2afa7c324e03aa24b28063eae728 reported cstate change: term changed from 0 to 1, leader changed from <none> to 970e2afa7c324e03aa24b28063eae728 (127.18.45.195). New cstate: current_term: 1 leader_uuid: "970e2afa7c324e03aa24b28063eae728" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f103722ba11f441dbda1143461876f35" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "970e2afa7c324e03aa24b28063eae728" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 36453 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c4699a06f354c3db12e2b056bd7ccfb" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 39517 } health_report { overall_health: UNKNOWN } } }
I20260806 22:47:42.179999 7441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:42.177501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43506 (local address 127.18.45.195:36453)
0806 22:47:42.177564 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:42.177566 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:42.177575 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:47:42.177654 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:42.177655 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:42.177675 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:42.177741 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:47:42.177856 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:42.178393 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:42.179218 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:42.179439 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:42.179500 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:42.179669 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:47:42.179784 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:47:42.179847 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:42.179881 (+ 34us) server_negotiation.cc:299] Negotiation successful
0806 22:47:42.179900 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 22:47:42.182386 7199 raft_consensus.cc:1261] T 2c62a5ffa9e24703bcb0568cd43588f8 P f103722ba11f441dbda1143461876f35 [term 1 FOLLOWER]: Refusing update from remote peer 970e2afa7c324e03aa24b28063eae728: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:47:42.182385 7011 raft_consensus.cc:1261] T 2c62a5ffa9e24703bcb0568cd43588f8 P 3c4699a06f354c3db12e2b056bd7ccfb [term 1 FOLLOWER]: Refusing update from remote peer 970e2afa7c324e03aa24b28063eae728: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:47:42.182794 7453 consensus_queue.cc:1035] T 2c62a5ffa9e24703bcb0568cd43588f8 P 970e2afa7c324e03aa24b28063eae728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f103722ba11f441dbda1143461876f35" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:47:42.182991 7456 consensus_queue.cc:1035] T 2c62a5ffa9e24703bcb0568cd43588f8 P 970e2afa7c324e03aa24b28063eae728 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c4699a06f354c3db12e2b056bd7ccfb" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 39517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:47:42.187307 7462 mvcc.cc:205] Tried to move back new op lower bound from 7315687269095383040 to 7315687269051764736. Current Snapshot: MvccSnapshot[applied={T|T < 7315687269095383040}]
I20260806 22:47:42.188835 6821 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:35674:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:47:42.188978 6821 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:35674:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:47:42.189881 6821 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a6658b79b3ec4a41ab783c6eb6a8f1cc: Sending DeleteTablet for 3 replicas of tablet 2c62a5ffa9e24703bcb0568cd43588f8
I20260806 22:47:42.190290 7341 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c62a5ffa9e24703bcb0568cd43588f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:47:42 UTC) from {username='kudu', principal='kudu/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:43504
I20260806 22:47:42.190307 6982 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c62a5ffa9e24703bcb0568cd43588f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:47:42 UTC) from {username='kudu', principal='kudu/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:59522
I20260806 22:47:42.190306 7169 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c62a5ffa9e24703bcb0568cd43588f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:47:42 UTC) from {username='kudu', principal='kudu/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:54344
I20260806 22:47:42.190431 7088 tablet_replica.cc:330] T 2c62a5ffa9e24703bcb0568cd43588f8 P 3c4699a06f354c3db12e2b056bd7ccfb: stopping tablet replica
I20260806 22:47:42.190493 7088 raft_consensus.cc:2229] T 2c62a5ffa9e24703bcb0568cd43588f8 P 3c4699a06f354c3db12e2b056bd7ccfb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:47:42.190564 7088 raft_consensus.cc:2258] T 2c62a5ffa9e24703bcb0568cd43588f8 P 3c4699a06f354c3db12e2b056bd7ccfb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:47:42.190711 7473 tablet_replica.cc:330] T 2c62a5ffa9e24703bcb0568cd43588f8 P f103722ba11f441dbda1143461876f35: stopping tablet replica
I20260806 22:47:42.190719 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 6874
I20260806 22:47:42.190820 7473 raft_consensus.cc:2229] T 2c62a5ffa9e24703bcb0568cd43588f8 P f103722ba11f441dbda1143461876f35 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:47:42.190959 7473 raft_consensus.cc:2258] T 2c62a5ffa9e24703bcb0568cd43588f8 P f103722ba11f441dbda1143461876f35 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:47:42.191176 7088 ts_tablet_manager.cc:1905] T 2c62a5ffa9e24703bcb0568cd43588f8 P 3c4699a06f354c3db12e2b056bd7ccfb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:47:42.191416 7472 tablet_replica.cc:330] T 2c62a5ffa9e24703bcb0568cd43588f8 P 970e2afa7c324e03aa24b28063eae728: stopping tablet replica
I20260806 22:47:42.191506 7472 raft_consensus.cc:2229] T 2c62a5ffa9e24703bcb0568cd43588f8 P 970e2afa7c324e03aa24b28063eae728 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:47:42.191593 7472 raft_consensus.cc:2258] T 2c62a5ffa9e24703bcb0568cd43588f8 P 970e2afa7c324e03aa24b28063eae728 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:47:42.196661 7472 ts_tablet_manager.cc:1905] T 2c62a5ffa9e24703bcb0568cd43588f8 P 970e2afa7c324e03aa24b28063eae728: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:47:42.198709 7472 ts_tablet_manager.cc:1918] T 2c62a5ffa9e24703bcb0568cd43588f8 P 970e2afa7c324e03aa24b28063eae728: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:47:42.198858 7472 log.cc:1196] T 2c62a5ffa9e24703bcb0568cd43588f8 P 970e2afa7c324e03aa24b28063eae728: Deleting WAL directory at /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-2/wal/wals/2c62a5ffa9e24703bcb0568cd43588f8
I20260806 22:47:42.199388 7472 ts_tablet_manager.cc:1939] T 2c62a5ffa9e24703bcb0568cd43588f8 P 970e2afa7c324e03aa24b28063eae728: Deleting consensus metadata
I20260806 22:47:42.204850 6803 catalog_manager.cc:4841] TS 970e2afa7c324e03aa24b28063eae728 (127.18.45.195:36453): tablet 2c62a5ffa9e24703bcb0568cd43588f8 (table test-table [id=ca9f3f42d9424c94ab2be7d8993609be]) successfully deleted
I20260806 22:47:42.207551 7473 ts_tablet_manager.cc:1905] T 2c62a5ffa9e24703bcb0568cd43588f8 P f103722ba11f441dbda1143461876f35: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:47:42.209798 7473 ts_tablet_manager.cc:1918] T 2c62a5ffa9e24703bcb0568cd43588f8 P f103722ba11f441dbda1143461876f35: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:47:42.209916 7473 log.cc:1196] T 2c62a5ffa9e24703bcb0568cd43588f8 P f103722ba11f441dbda1143461876f35: Deleting WAL directory at /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-1/wal/wals/2c62a5ffa9e24703bcb0568cd43588f8
I20260806 22:47:42.210274 7473 ts_tablet_manager.cc:1939] T 2c62a5ffa9e24703bcb0568cd43588f8 P f103722ba11f441dbda1143461876f35: Deleting consensus metadata
I20260806 22:47:42.211081 6802 catalog_manager.cc:4841] TS f103722ba11f441dbda1143461876f35 (127.18.45.194:33871): tablet 2c62a5ffa9e24703bcb0568cd43588f8 (table test-table [id=ca9f3f42d9424c94ab2be7d8993609be]) successfully deleted
W20260806 22:47:42.211839 6802 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.45.193:39517 (error 108)
I20260806 22:47:42.212265 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 7084
I20260806 22:47:42.212502 7440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:42.212341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.193:39517 (local address 127.0.0.1:59526)
0806 22:47:42.212408 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:42.212450 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.193:39517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260806 22:47:42.223327 6802 catalog_manager.cc:4584] TS 3c4699a06f354c3db12e2b056bd7ccfb (127.18.45.193:39517): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2c62a5ffa9e24703bcb0568cd43588f8: Network error: Client connection negotiation failed: client connection to 127.18.45.193:39517: connect: Connection refused (error 111)
I20260806 22:47:42.226671 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 7265
I20260806 22:47:42.238384 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 6776
I20260806 22:47:42.255797 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:39797
--webserver_interface=127.18.45.254
--webserver_port=35941
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:37755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.45.254:39797
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:47:42.353441 7475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:42.353753 7475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:42.353796 7475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:42.355105 7475 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:47:42.355181 7475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:42.355206 7475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:42.355235 7475 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:47:42.355266 7475 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:47:42.356884 7475 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:37755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.45.254:39797
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:39797
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.45.254
--webserver_port=35941
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7475
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:47:42.357223 7475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:42.357496 7475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:42.363057 7482 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:42.363082 7481 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:42.363093 7484 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:42.363651 7475 server_base.cc:1034] running on GCE node
I20260806 22:47:42.363991 7475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:42.364341 7475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:42.365516 7475 hybrid_clock.cc:648] HybridClock initialized: now 1786056462365483 us; error 54 us; skew 500 ppm
Aug 06 22:47:42 dist-test-slave-h3ft krb5kdc[6039](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056462, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:42.368686 7475 init.cc:377] Logged in from keytab as kudu/127.18.45.254@KRBTEST.COM (short username kudu)
I20260806 22:47:42.369275 7475 webserver.cc:460] Webserver started at http://127.18.45.254:35941/ using document root <none> and password file <none>
I20260806 22:47:42.369570 7475 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:42.369648 7475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:42.371248 7475 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:47:42.372186 7494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:42.372434 7475 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:47:42.372583 7475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/master-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/master-0/wal
uuid: "a6658b79b3ec4a41ab783c6eb6a8f1cc"
format_stamp: "Formatted at 2026-08-06 22:47:40 on dist-test-slave-h3ft"
server_key: "10442fe2d6f5b61f368ac8bed514eec1"
server_key_iv: "2bb5c76b1bbe99b2b3eba2b2052f2fa0"
server_key_version: "encryptionkey@0"
I20260806 22:47:42.373056 7475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:42.382450 7475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:42.385301 7475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:42.385505 7475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:42.391191 7475 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.254:39797
I20260806 22:47:42.391273 7560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.254:39797 every 8 connection(s)
I20260806 22:47:42.391762 7475 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
I20260806 22:47:42.393149 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 7475
I20260806 22:47:42.393549 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.193:39517
--local_ip_for_outbound_sockets=127.18.45.193
--tserver_master_addrs=127.18.45.254:39797
--webserver_port=44663
--webserver_interface=127.18.45.193
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab
--principal=kudu/127.18.45.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:37755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:47:42.394408 7561 sys_catalog.cc:263] Verifying existing consensus state
I20260806 22:47:42.395725 7561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6658b79b3ec4a41ab783c6eb6a8f1cc: Bootstrap starting.
I20260806 22:47:42.408226 7561 log.cc:824] T 00000000000000000000000000000000 P a6658b79b3ec4a41ab783c6eb6a8f1cc: Log is configured to *not* fsync() on all Append() calls
I20260806 22:47:42.411434 7561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6658b79b3ec4a41ab783c6eb6a8f1cc: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:47:42.411850 7561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6658b79b3ec4a41ab783c6eb6a8f1cc: Bootstrap complete.
I20260806 22:47:42.414673 7561 raft_consensus.cc:348] T 00000000000000000000000000000000 P a6658b79b3ec4a41ab783c6eb6a8f1cc [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6658b79b3ec4a41ab783c6eb6a8f1cc" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39797 } }
I20260806 22:47:42.414989 7561 raft_consensus.cc:729] T 00000000000000000000000000000000 P a6658b79b3ec4a41ab783c6eb6a8f1cc [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6658b79b3ec4a41ab783c6eb6a8f1cc, State: Initialized, Role: FOLLOWER
I20260806 22:47:42.415274 7561 consensus_queue.cc:260] T 00000000000000000000000000000000 P a6658b79b3ec4a41ab783c6eb6a8f1cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6658b79b3ec4a41ab783c6eb6a8f1cc" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39797 } }
I20260806 22:47:42.415351 7561 raft_consensus.cc:388] T 00000000000000000000000000000000 P a6658b79b3ec4a41ab783c6eb6a8f1cc [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:47:42.415426 7561 raft_consensus.cc:482] T 00000000000000000000000000000000 P a6658b79b3ec4a41ab783c6eb6a8f1cc [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:47:42.415480 7561 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a6658b79b3ec4a41ab783c6eb6a8f1cc [term 2 FOLLOWER]: Advancing to term 3
I20260806 22:47:42.416745 7561 raft_consensus.cc:504] T 00000000000000000000000000000000 P a6658b79b3ec4a41ab783c6eb6a8f1cc [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6658b79b3ec4a41ab783c6eb6a8f1cc" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39797 } }
I20260806 22:47:42.416849 7561 leader_election.cc:302] T 00000000000000000000000000000000 P a6658b79b3ec4a41ab783c6eb6a8f1cc [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a6658b79b3ec4a41ab783c6eb6a8f1cc; no voters:
I20260806 22:47:42.417129 7561 leader_election.cc:288] T 00000000000000000000000000000000 P a6658b79b3ec4a41ab783c6eb6a8f1cc [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 22:47:42.417176 7568 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a6658b79b3ec4a41ab783c6eb6a8f1cc [term 3 FOLLOWER]: Leader election won for term 3
I20260806 22:47:42.417320 7568 raft_consensus.cc:686] T 00000000000000000000000000000000 P a6658b79b3ec4a41ab783c6eb6a8f1cc [term 3 LEADER]: Becoming Leader. State: Replica: a6658b79b3ec4a41ab783c6eb6a8f1cc, State: Running, Role: LEADER
I20260806 22:47:42.417433 7568 consensus_queue.cc:237] T 00000000000000000000000000000000 P a6658b79b3ec4a41ab783c6eb6a8f1cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6658b79b3ec4a41ab783c6eb6a8f1cc" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39797 } }
I20260806 22:47:42.417481 7561 sys_catalog.cc:564] T 00000000000000000000000000000000 P a6658b79b3ec4a41ab783c6eb6a8f1cc [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:47:42.417760 7568 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6658b79b3ec4a41ab783c6eb6a8f1cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader a6658b79b3ec4a41ab783c6eb6a8f1cc. Latest consensus state: current_term: 3 leader_uuid: "a6658b79b3ec4a41ab783c6eb6a8f1cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6658b79b3ec4a41ab783c6eb6a8f1cc" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39797 } } }
I20260806 22:47:42.417847 7568 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6658b79b3ec4a41ab783c6eb6a8f1cc [sys.catalog]: This master's current role is: LEADER
I20260806 22:47:42.417768 7569 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6658b79b3ec4a41ab783c6eb6a8f1cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "a6658b79b3ec4a41ab783c6eb6a8f1cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6658b79b3ec4a41ab783c6eb6a8f1cc" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39797 } } }
I20260806 22:47:42.418051 7569 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6658b79b3ec4a41ab783c6eb6a8f1cc [sys.catalog]: This master's current role is: LEADER
I20260806 22:47:42.418311 7575 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:47:42.419405 7575 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:47:42.419552 7575 catalog_manager.cc:1259] Loaded cluster ID: 821df2c33fa14fca8c4df6091d48ed3e
I20260806 22:47:42.419591 7575 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260806 22:47:42.419839 7575 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f83e7f789ad google::LogMessage::Fail() at ??:0
@ 0x7f83e7f7b88a google::LogMessage::SendToLog() at ??:0
@ 0x7f83e7f784de google::LogMessage::Flush() at ??:0
@ 0x7f83e7f79a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f83efe9e0f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f83efea78b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f83e9e460d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f83e9e3b954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f83eac9f6db start_thread at ??:0
@ 0x7f83e6a7661f clone at ??:0
W20260806 22:47:42.528419 7564 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:42.528784 7564 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:42.528832 7564 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:42.530745 7564 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:42.530812 7564 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:42.530857 7564 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.193
I20260806 22:47:42.533241 7564 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:37755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab
--principal=kudu/127.18.45.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.193:39517
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.45.193
--webserver_port=44663
--tserver_master_addrs=127.18.45.254:39797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7564
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.193
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:47:42.533562 7564 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:42.533865 7564 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:42.540277 7612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:42.540432 7615 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:42.540544 7613 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:42.540571 7564 server_base.cc:1034] running on GCE node
I20260806 22:47:42.540917 7564 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:42.541250 7564 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:42.542445 7564 hybrid_clock.cc:648] HybridClock initialized: now 1786056462542410 us; error 51 us; skew 500 ppm
Aug 06 22:47:42 dist-test-slave-h3ft krb5kdc[6039](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056462, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:42.546402 7564 init.cc:377] Logged in from keytab as kudu/127.18.45.193@KRBTEST.COM (short username kudu)
I20260806 22:47:42.547081 7564 webserver.cc:460] Webserver started at http://127.18.45.193:44663/ using document root <none> and password file <none>
I20260806 22:47:42.547386 7564 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:42.547457 7564 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:42.549583 7564 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 22:47:42.550711 7624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:42.550926 7564 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:47:42.551051 7564 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-0/wal
uuid: "3c4699a06f354c3db12e2b056bd7ccfb"
format_stamp: "Formatted at 2026-08-06 22:47:40 on dist-test-slave-h3ft"
server_key: "ab901a5d6a2afc33687d31cb5e9d5235"
server_key_iv: "61bc0e5d62033c27349220873c9abc5d"
server_key_version: "encryptionkey@0"
I20260806 22:47:42.551630 7564 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:42.575609 7564 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:42.578812 7564 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:42.579023 7564 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:42.579396 7564 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:47:42.580104 7632 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 22:47:42.582733 7564 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 22:47:42.582808 7564 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.000s
I20260806 22:47:42.582851 7564 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 22:47:42.583982 7564 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 22:47:42.584030 7564 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260806 22:47:42.584240 7632 tablet_bootstrap.cc:492] T 2c62a5ffa9e24703bcb0568cd43588f8 P 3c4699a06f354c3db12e2b056bd7ccfb: Bootstrap starting.
I20260806 22:47:42.586930 7632 log.cc:824] T 2c62a5ffa9e24703bcb0568cd43588f8 P 3c4699a06f354c3db12e2b056bd7ccfb: Log is configured to *not* fsync() on all Append() calls
I20260806 22:47:42.591079 7632 tablet_bootstrap.cc:492] T 2c62a5ffa9e24703bcb0568cd43588f8 P 3c4699a06f354c3db12e2b056bd7ccfb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:47:42.591567 7632 tablet_bootstrap.cc:492] T 2c62a5ffa9e24703bcb0568cd43588f8 P 3c4699a06f354c3db12e2b056bd7ccfb: Bootstrap complete.
I20260806 22:47:42.591832 7632 ts_tablet_manager.cc:1397] T 2c62a5ffa9e24703bcb0568cd43588f8 P 3c4699a06f354c3db12e2b056bd7ccfb: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260806 22:47:42.593703 7564 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.193:39517
I20260806 22:47:42.593748 7786 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.193:39517 every 8 connection(s)
I20260806 22:47:42.594146 7564 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
I20260806 22:47:42.594712 7632 raft_consensus.cc:348] T 2c62a5ffa9e24703bcb0568cd43588f8 P 3c4699a06f354c3db12e2b056bd7ccfb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f103722ba11f441dbda1143461876f35" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33871 } } peers { permanent_uuid: "970e2afa7c324e03aa24b28063eae728" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 36453 } } peers { permanent_uuid: "3c4699a06f354c3db12e2b056bd7ccfb" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 39517 } }
I20260806 22:47:42.595779 7632 raft_consensus.cc:729] T 2c62a5ffa9e24703bcb0568cd43588f8 P 3c4699a06f354c3db12e2b056bd7ccfb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c4699a06f354c3db12e2b056bd7ccfb, State: Initialized, Role: FOLLOWER
I20260806 22:47:42.595983 7632 consensus_queue.cc:260] T 2c62a5ffa9e24703bcb0568cd43588f8 P 3c4699a06f354c3db12e2b056bd7ccfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f103722ba11f441dbda1143461876f35" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33871 } } peers { permanent_uuid: "970e2afa7c324e03aa24b28063eae728" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 36453 } } peers { permanent_uuid: "3c4699a06f354c3db12e2b056bd7ccfb" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 39517 } }
I20260806 22:47:42.596308 7632 ts_tablet_manager.cc:1428] T 2c62a5ffa9e24703bcb0568cd43588f8 P 3c4699a06f354c3db12e2b056bd7ccfb: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260806 22:47:42.602357 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 7564
I20260806 22:47:42.602743 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.194:33871
--local_ip_for_outbound_sockets=127.18.45.194
--tserver_master_addrs=127.18.45.254:39797
--webserver_port=39845
--webserver_interface=127.18.45.194
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab
--principal=kudu/127.18.45.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:37755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:47:42.692525 7790 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:47:42.595032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39797 (local address 127.18.45.193:37635)
0806 22:47:42.595241 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:42.595297 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:42.595488 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:42.692231 (+ 96743us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39797: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":108,"threads_started":1}
W20260806 22:47:42.692907 7787 heartbeater.cc:644] Failed to heartbeat to 127.18.45.254:39797 (0 consecutive failures): Network error: Failed to ping master at 127.18.45.254:39797: Client connection negotiation failed: client connection to 127.18.45.254:39797: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 22:47:42.693334 7790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:42.693195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39797 (local address 127.18.45.193:43239)
0806 22:47:42.693261 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:42.693312 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260806 22:47:42.693753 7790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:42.693629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39797 (local address 127.18.45.193:56325)
0806 22:47:42.693679 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:42.693735 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260806 22:47:42.693884 7787 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 22:47:42.721709 7792 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:42.721971 7792 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:42.722000 7792 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:42.723248 7792 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:42.723290 7792 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:42.723317 7792 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.194
I20260806 22:47:42.724810 7792 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:37755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab
--principal=kudu/127.18.45.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.194:33871
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.45.194
--webserver_port=39845
--tserver_master_addrs=127.18.45.254:39797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7792
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.194
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:47:42.725057 7792 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:42.725281 7792 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:42.731343 7803 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:42.731352 7801 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:42.731575 7792 server_base.cc:1034] running on GCE node
W20260806 22:47:42.731367 7800 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:42.732048 7792 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:42.732383 7792 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:42.733577 7792 hybrid_clock.cc:648] HybridClock initialized: now 1786056462733558 us; error 41 us; skew 500 ppm
Aug 06 22:47:42 dist-test-slave-h3ft krb5kdc[6039](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056462, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:42.736867 7792 init.cc:377] Logged in from keytab as kudu/127.18.45.194@KRBTEST.COM (short username kudu)
I20260806 22:47:42.737349 7792 webserver.cc:460] Webserver started at http://127.18.45.194:39845/ using document root <none> and password file <none>
I20260806 22:47:42.737569 7792 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:42.737617 7792 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:42.739223 7792 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:47:42.739971 7815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:42.740218 7792 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:47:42.740310 7792 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-1/wal
uuid: "f103722ba11f441dbda1143461876f35"
format_stamp: "Formatted at 2026-08-06 22:47:40 on dist-test-slave-h3ft"
server_key: "92f81981150686adb8d499e315f3f9bb"
server_key_iv: "1b051bfa9ea92fc75a91a60048c980ac"
server_key_version: "encryptionkey@0"
I20260806 22:47:42.740737 7792 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:42.757380 7792 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:42.760327 7792 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:42.760483 7792 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:42.760747 7792 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:47:42.761152 7792 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:47:42.761193 7792 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:42.761221 7792 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:47:42.761234 7792 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:42.768479 7792 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.194:33871
I20260806 22:47:42.768589 7954 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.194:33871 every 8 connection(s)
I20260806 22:47:42.769011 7792 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
I20260806 22:47:42.770450 7959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:42.769907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39797 (local address 127.18.45.194:40959)
0806 22:47:42.770152 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:42.770263 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":81,"threads_started":1}
W20260806 22:47:42.770957 7956 heartbeater.cc:644] Failed to heartbeat to 127.18.45.254:39797 (0 consecutive failures): Network error: Failed to ping master at 127.18.45.254:39797: Client connection negotiation failed: client connection to 127.18.45.254:39797: connect: Connection refused (error 111)
I20260806 22:47:42.771364 7959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:42.771253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39797 (local address 127.18.45.194:33381)
0806 22:47:42.771306 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:42.771341 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 22:47:42.771659 7959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:42.771605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39797 (local address 127.18.45.194:41043)
0806 22:47:42.771626 (+ 21us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:42.771646 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260806 22:47:42.771786 7956 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 22:47:42.772262 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 7792
I20260806 22:47:42.772503 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.195:36453
--local_ip_for_outbound_sockets=127.18.45.195
--tserver_master_addrs=127.18.45.254:39797
--webserver_port=44609
--webserver_interface=127.18.45.195
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab
--principal=kudu/127.18.45.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:37755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:47:42.874871 7961 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:42.875222 7961 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:42.875257 7961 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:42.876543 7961 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:42.876587 7961 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:42.876614 7961 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.195
I20260806 22:47:42.878194 7961 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:37755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab
--principal=kudu/127.18.45.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.195:36453
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.45.195
--webserver_port=44609
--tserver_master_addrs=127.18.45.254:39797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7961
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.195
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:47:42.878438 7961 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:42.878669 7961 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:42.884779 7967 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:42.884789 7966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:42.884795 7970 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:42.885076 7961 server_base.cc:1034] running on GCE node
I20260806 22:47:42.885349 7961 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:42.885602 7961 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:42.886763 7961 hybrid_clock.cc:648] HybridClock initialized: now 1786056462886752 us; error 36 us; skew 500 ppm
Aug 06 22:47:42 dist-test-slave-h3ft krb5kdc[6039](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056462, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:42.889942 7961 init.cc:377] Logged in from keytab as kudu/127.18.45.195@KRBTEST.COM (short username kudu)
I20260806 22:47:42.890542 7961 webserver.cc:460] Webserver started at http://127.18.45.195:44609/ using document root <none> and password file <none>
I20260806 22:47:42.890820 7961 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:42.890865 7961 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:42.892498 7961 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:47:42.893396 7987 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:42.893708 7961 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:47:42.893846 7961 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-2/wal
uuid: "970e2afa7c324e03aa24b28063eae728"
format_stamp: "Formatted at 2026-08-06 22:47:40 on dist-test-slave-h3ft"
server_key: "1b0acecf3389179e2102ca862d630c30"
server_key_iv: "6cb2d268c209228a8f4b607284fa7c66"
server_key_version: "encryptionkey@0"
I20260806 22:47:42.894299 7961 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:42.922992 7961 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:42.926141 7961 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:42.926344 7961 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:42.926709 7961 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:47:42.927206 7961 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:47:42.927274 7961 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:42.927316 7961 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:47:42.927340 7961 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:42.935685 7961 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.195:36453
I20260806 22:47:42.935808 8157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.195:36453 every 8 connection(s)
I20260806 22:47:42.936205 7961 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
I20260806 22:47:42.937631 8161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:42.937135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39797 (local address 127.18.45.195:55535)
0806 22:47:42.937335 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:42.937435 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":93,"threads_started":1}
W20260806 22:47:42.938191 8158 heartbeater.cc:644] Failed to heartbeat to 127.18.45.254:39797 (0 consecutive failures): Network error: Failed to ping master at 127.18.45.254:39797: Client connection negotiation failed: client connection to 127.18.45.254:39797: connect: Connection refused (error 111)
I20260806 22:47:42.938552 8161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:42.938452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39797 (local address 127.18.45.195:52245)
0806 22:47:42.938500 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:42.938531 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260806 22:47:42.939126 8161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:42.938932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39797 (local address 127.18.45.195:49125)
0806 22:47:42.939056 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:42.939091 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260806 22:47:42.939303 8158 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 22:47:42.941419 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 7961
I20260806 22:47:43.694700 8163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:43.694357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39797 (local address 127.18.45.193:51863)
0806 22:47:43.694548 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:43.694602 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":86,"threads_started":1}
I20260806 22:47:43.772549 8164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:43.772214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39797 (local address 127.18.45.194:57173)
0806 22:47:43.772423 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:43.772474 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":125,"threads_started":1}
I20260806 22:47:43.940027 8165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:43.939708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39797 (local address 127.18.45.195:59473)
0806 22:47:43.939870 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:43.939953 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":77,"threads_started":1}
I20260806 22:47:44.202700 8166 raft_consensus.cc:482] T 2c62a5ffa9e24703bcb0568cd43588f8 P 3c4699a06f354c3db12e2b056bd7ccfb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:47:44.202919 8166 raft_consensus.cc:504] T 2c62a5ffa9e24703bcb0568cd43588f8 P 3c4699a06f354c3db12e2b056bd7ccfb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f103722ba11f441dbda1143461876f35" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33871 } } peers { permanent_uuid: "970e2afa7c324e03aa24b28063eae728" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 36453 } } peers { permanent_uuid: "3c4699a06f354c3db12e2b056bd7ccfb" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 39517 } }
I20260806 22:47:44.203445 8166 leader_election.cc:288] T 2c62a5ffa9e24703bcb0568cd43588f8 P 3c4699a06f354c3db12e2b056bd7ccfb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f103722ba11f441dbda1143461876f35 (127.18.45.194:33871), 970e2afa7c324e03aa24b28063eae728 (127.18.45.195:36453)
Aug 06 22:47:44 dist-test-slave-h3ft krb5kdc[6039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056462, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.193@KRBTEST.COM for kudu/127.18.45.195@KRBTEST.COM
Aug 06 22:47:44 dist-test-slave-h3ft krb5kdc[6039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056462, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.193@KRBTEST.COM for kudu/127.18.45.194@KRBTEST.COM
I20260806 22:47:44.218799 8172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:44.203949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:36239 (local address 127.18.45.194:33871)
0806 22:47:44.204131 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:44.204138 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:44.204195 (+ 57us) server_negotiation.cc:407] Connection header received
0806 22:47:44.204887 (+ 692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:44.204891 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:44.204959 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:44.205154 (+ 195us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:44.206470 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:44.208064 (+ 1594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:44.209812 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:44.210491 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:44.210625 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:44.214377 (+ 3752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:44.214380 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:44.214380 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:44.214394 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:44.216512 (+ 2118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:44.217172 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:44.217175 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:44.217176 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:44.217318 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:44.217670 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:44.217671 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:44.217671 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:44.218035 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:44.218121 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:44.218511 (+ 390us) server_negotiation.cc:299] Negotiation successful
0806 22:47:44.218660 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":86,"threads_started":1}
I20260806 22:47:44.218799 8171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:44.203832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:46835 (local address 127.18.45.195:36453)
0806 22:47:44.204076 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:44.204092 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:44.204135 (+ 43us) server_negotiation.cc:407] Connection header received
0806 22:47:44.204887 (+ 752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:44.204890 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:44.204959 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:44.205154 (+ 195us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:44.206683 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:44.208045 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:44.209812 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:44.210491 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:44.210626 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:44.214191 (+ 3565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:44.214196 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:44.214197 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:44.214210 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:44.216523 (+ 2313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:44.217172 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:44.217175 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:44.217176 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:44.217318 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:44.217670 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:44.217671 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:44.217671 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:44.218028 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:44.218121 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:44.218518 (+ 397us) server_negotiation.cc:299] Negotiation successful
0806 22:47:44.218661 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":94,"threads_started":1}
I20260806 22:47:44.218819 8168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:44.203627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.194:33871 (local address 127.18.45.193:36239)
0806 22:47:44.204129 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:44.204139 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:44.204205 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:44.205271 (+ 1066us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:44.205275 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:44.205885 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:44.206309 (+ 424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:44.206321 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:44.208194 (+ 1873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:44.208196 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:44.209710 (+ 1514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:44.209714 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:44.210636 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:44.210638 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:44.210813 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:44.211593 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:44.211607 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:44.214267 (+ 2660us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:44.216608 (+ 2341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:44.216619 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:44.216622 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:44.217077 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:44.217396 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:44.217396 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:44.217397 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:44.217592 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:44.218109 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:44.218118 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:44.218308 (+ 190us) client_negotiation.cc:769] Sending connection context
0806 22:47:44.218436 (+ 128us) client_negotiation.cc:240] Negotiation successful
0806 22:47:44.218672 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"mutex_wait_us":89,"thread_start_us":80,"threads_started":1}
I20260806 22:47:44.218833 8167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:44.203618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.195:36453 (local address 127.18.45.193:46835)
0806 22:47:44.203787 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:44.203802 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:44.203880 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:44.205271 (+ 1391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:44.205275 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:44.205885 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:44.206563 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:44.206566 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:44.208196 (+ 1630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:44.208197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:44.209710 (+ 1513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:44.209714 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:44.210639 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:44.210640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:44.210813 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:44.211609 (+ 796us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:44.211614 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:44.214057 (+ 2443us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:44.216610 (+ 2553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:44.216619 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:44.216622 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:44.217077 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:44.217396 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:44.217396 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:44.217397 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:44.217592 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:44.218109 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:44.218118 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:44.218308 (+ 190us) client_negotiation.cc:769] Sending connection context
0806 22:47:44.218436 (+ 128us) client_negotiation.cc:240] Negotiation successful
0806 22:47:44.218726 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"mutex_wait_us":304,"thread_start_us":89,"threads_started":1}
W20260806 22:47:44.219872 7628 leader_election.cc:341] T 2c62a5ffa9e24703bcb0568cd43588f8 P 3c4699a06f354c3db12e2b056bd7ccfb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 970e2afa7c324e03aa24b28063eae728 (127.18.45.195:36453): Not found: Tablet not found: 2c62a5ffa9e24703bcb0568cd43588f8
W20260806 22:47:44.220434 7627 leader_election.cc:341] T 2c62a5ffa9e24703bcb0568cd43588f8 P 3c4699a06f354c3db12e2b056bd7ccfb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f103722ba11f441dbda1143461876f35 (127.18.45.194:33871): Not found: Tablet not found: 2c62a5ffa9e24703bcb0568cd43588f8
I20260806 22:47:44.220536 7627 leader_election.cc:302] T 2c62a5ffa9e24703bcb0568cd43588f8 P 3c4699a06f354c3db12e2b056bd7ccfb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c4699a06f354c3db12e2b056bd7ccfb; no voters: 970e2afa7c324e03aa24b28063eae728, f103722ba11f441dbda1143461876f35
I20260806 22:47:44.220755 8166 raft_consensus.cc:2726] T 2c62a5ffa9e24703bcb0568cd43588f8 P 3c4699a06f354c3db12e2b056bd7ccfb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 22:47:44.695524 8167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:44.695323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39797 (local address 127.18.45.193:60799)
0806 22:47:44.695397 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:44.695445 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260806 22:47:44.773486 8173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:44.773198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39797 (local address 127.18.45.194:37711)
0806 22:47:44.773359 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:44.773415 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":82,"threads_started":1}
I20260806 22:47:44.941054 8174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:44.940678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39797 (local address 127.18.45.195:48127)
0806 22:47:44.940909 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:44.940962 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":131,"threads_started":1}
I20260806 22:47:45.696666 8175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:45.696327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39797 (local address 127.18.45.193:54953)
0806 22:47:45.696536 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:45.696593 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":95,"threads_started":1}
I20260806 22:47:45.774555 8176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:45.774201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39797 (local address 127.18.45.194:41285)
0806 22:47:45.774406 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:45.774481 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":98,"threads_started":1}
I20260806 22:47:45.942094 8177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:45.941766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39797 (local address 127.18.45.195:54607)
0806 22:47:45.941966 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:45.942023 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":97,"threads_started":1}
I20260806 22:47:45.945986 8179 raft_consensus.cc:482] T 2c62a5ffa9e24703bcb0568cd43588f8 P 3c4699a06f354c3db12e2b056bd7ccfb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:47:45.946106 8179 raft_consensus.cc:504] T 2c62a5ffa9e24703bcb0568cd43588f8 P 3c4699a06f354c3db12e2b056bd7ccfb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f103722ba11f441dbda1143461876f35" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33871 } } peers { permanent_uuid: "970e2afa7c324e03aa24b28063eae728" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 36453 } } peers { permanent_uuid: "3c4699a06f354c3db12e2b056bd7ccfb" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 39517 } }
I20260806 22:47:45.946492 8179 leader_election.cc:288] T 2c62a5ffa9e24703bcb0568cd43588f8 P 3c4699a06f354c3db12e2b056bd7ccfb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f103722ba11f441dbda1143461876f35 (127.18.45.194:33871), 970e2afa7c324e03aa24b28063eae728 (127.18.45.195:36453)
W20260806 22:47:45.947037 7628 leader_election.cc:341] T 2c62a5ffa9e24703bcb0568cd43588f8 P 3c4699a06f354c3db12e2b056bd7ccfb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 970e2afa7c324e03aa24b28063eae728 (127.18.45.195:36453): Not found: Tablet not found: 2c62a5ffa9e24703bcb0568cd43588f8
W20260806 22:47:45.947151 7627 leader_election.cc:341] T 2c62a5ffa9e24703bcb0568cd43588f8 P 3c4699a06f354c3db12e2b056bd7ccfb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f103722ba11f441dbda1143461876f35 (127.18.45.194:33871): Not found: Tablet not found: 2c62a5ffa9e24703bcb0568cd43588f8
I20260806 22:47:45.947230 7627 leader_election.cc:302] T 2c62a5ffa9e24703bcb0568cd43588f8 P 3c4699a06f354c3db12e2b056bd7ccfb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c4699a06f354c3db12e2b056bd7ccfb; no voters: 970e2afa7c324e03aa24b28063eae728, f103722ba11f441dbda1143461876f35
I20260806 22:47:45.947314 8179 raft_consensus.cc:2726] T 2c62a5ffa9e24703bcb0568cd43588f8 P 3c4699a06f354c3db12e2b056bd7ccfb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 22:47:46.697789 8181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:46.697377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39797 (local address 127.18.45.193:42995)
0806 22:47:46.697639 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:46.697710 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":123,"threads_started":1}
I20260806 22:47:46.775592 8183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:46.775291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39797 (local address 127.18.45.194:46185)
0806 22:47:46.775461 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:46.775522 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":86,"threads_started":1}
I20260806 22:47:46.943374 8185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:46.942983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39797 (local address 127.18.45.195:58609)
0806 22:47:46.943246 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:46.943302 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":146,"threads_started":1}
I20260806 22:47:47.699220 8186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:47.698709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39797 (local address 127.18.45.193:43671)
0806 22:47:47.698953 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:47.699146 (+ 193us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":90,"threads_started":1}
I20260806 22:47:47.776616 8187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:47.776302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39797 (local address 127.18.45.194:33419)
0806 22:47:47.776490 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:47.776545 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":87,"threads_started":1}
I20260806 22:47:47.944474 8188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:47.944115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39797 (local address 127.18.45.195:42627)
0806 22:47:47.944339 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:47.944395 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":108,"threads_started":1}
I20260806 22:47:48.700260 8189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:48.699912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39797 (local address 127.18.45.193:52361)
0806 22:47:48.700145 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:48.700191 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":109,"threads_started":1}
I20260806 22:47:48.777510 8190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:48.777216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39797 (local address 127.18.45.194:42117)
0806 22:47:48.777384 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:48.777437 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":89,"threads_started":1}
I20260806 22:47:48.945629 8191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:48.945301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39797 (local address 127.18.45.195:54631)
0806 22:47:48.945502 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:48.945558 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":94,"threads_started":1}
I20260806 22:47:49.125670 8192 raft_consensus.cc:482] T 2c62a5ffa9e24703bcb0568cd43588f8 P 3c4699a06f354c3db12e2b056bd7ccfb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:47:49.125869 8192 raft_consensus.cc:504] T 2c62a5ffa9e24703bcb0568cd43588f8 P 3c4699a06f354c3db12e2b056bd7ccfb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f103722ba11f441dbda1143461876f35" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33871 } } peers { permanent_uuid: "970e2afa7c324e03aa24b28063eae728" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 36453 } } peers { permanent_uuid: "3c4699a06f354c3db12e2b056bd7ccfb" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 39517 } }
I20260806 22:47:49.126157 8192 leader_election.cc:288] T 2c62a5ffa9e24703bcb0568cd43588f8 P 3c4699a06f354c3db12e2b056bd7ccfb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f103722ba11f441dbda1143461876f35 (127.18.45.194:33871), 970e2afa7c324e03aa24b28063eae728 (127.18.45.195:36453)
W20260806 22:47:49.126726 7628 leader_election.cc:341] T 2c62a5ffa9e24703bcb0568cd43588f8 P 3c4699a06f354c3db12e2b056bd7ccfb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 970e2afa7c324e03aa24b28063eae728 (127.18.45.195:36453): Not found: Tablet not found: 2c62a5ffa9e24703bcb0568cd43588f8
W20260806 22:47:49.126828 7627 leader_election.cc:341] T 2c62a5ffa9e24703bcb0568cd43588f8 P 3c4699a06f354c3db12e2b056bd7ccfb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f103722ba11f441dbda1143461876f35 (127.18.45.194:33871): Not found: Tablet not found: 2c62a5ffa9e24703bcb0568cd43588f8
I20260806 22:47:49.126978 7628 leader_election.cc:302] T 2c62a5ffa9e24703bcb0568cd43588f8 P 3c4699a06f354c3db12e2b056bd7ccfb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c4699a06f354c3db12e2b056bd7ccfb; no voters: 970e2afa7c324e03aa24b28063eae728, f103722ba11f441dbda1143461876f35
I20260806 22:47:49.127102 8192 raft_consensus.cc:2726] T 2c62a5ffa9e24703bcb0568cd43588f8 P 3c4699a06f354c3db12e2b056bd7ccfb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 22:47:49.701383 8193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:49.701016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39797 (local address 127.18.45.193:39143)
0806 22:47:49.701250 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:49.701315 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":106,"threads_started":1}
I20260806 22:47:49.778386 8194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:49.778076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39797 (local address 127.18.45.194:37157)
0806 22:47:49.778267 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:49.778315 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":85,"threads_started":1}
I20260806 22:47:49.946731 8195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:49.946363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39797 (local address 127.18.45.195:48101)
0806 22:47:49.946575 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:49.946640 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":118,"threads_started":1}
I20260806 22:47:50.702464 8196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:50.702056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39797 (local address 127.18.45.193:56765)
0806 22:47:50.702295 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:50.702358 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":90,"threads_started":1}
I20260806 22:47:50.779338 8197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:50.779065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39797 (local address 127.18.45.194:40673)
0806 22:47:50.779219 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:50.779269 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":87,"threads_started":1}
I20260806 22:47:50.947675 8198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:50.947350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39797 (local address 127.18.45.195:52009)
0806 22:47:50.947549 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:50.947603 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":116,"threads_started":1}
I20260806 22:47:51.703545 8200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:51.703219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39797 (local address 127.18.45.193:44185)
0806 22:47:51.703415 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:51.703471 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":90,"threads_started":1}
I20260806 22:47:51.780275 8202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:51.779970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39797 (local address 127.18.45.194:40159)
0806 22:47:51.780149 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:51.780204 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":84,"threads_started":1}
I20260806 22:47:51.948530 8204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:51.948199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39797 (local address 127.18.45.195:47215)
0806 22:47:51.948410 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:51.948460 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":100,"threads_started":1}
I20260806 22:47:52.704627 8205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:52.704303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39797 (local address 127.18.45.193:36335)
0806 22:47:52.704491 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:52.704549 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":109,"threads_started":1}
I20260806 22:47:52.781303 8206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:52.780948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39797 (local address 127.18.45.194:38477)
0806 22:47:52.781176 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:52.781228 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":116,"threads_started":1}
I20260806 22:47:52.858122 8208 raft_consensus.cc:482] T 2c62a5ffa9e24703bcb0568cd43588f8 P 3c4699a06f354c3db12e2b056bd7ccfb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:47:52.858269 8208 raft_consensus.cc:504] T 2c62a5ffa9e24703bcb0568cd43588f8 P 3c4699a06f354c3db12e2b056bd7ccfb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f103722ba11f441dbda1143461876f35" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33871 } } peers { permanent_uuid: "970e2afa7c324e03aa24b28063eae728" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 36453 } } peers { permanent_uuid: "3c4699a06f354c3db12e2b056bd7ccfb" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 39517 } }
I20260806 22:47:52.858531 8208 leader_election.cc:288] T 2c62a5ffa9e24703bcb0568cd43588f8 P 3c4699a06f354c3db12e2b056bd7ccfb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f103722ba11f441dbda1143461876f35 (127.18.45.194:33871), 970e2afa7c324e03aa24b28063eae728 (127.18.45.195:36453)
W20260806 22:47:52.859081 7627 leader_election.cc:341] T 2c62a5ffa9e24703bcb0568cd43588f8 P 3c4699a06f354c3db12e2b056bd7ccfb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f103722ba11f441dbda1143461876f35 (127.18.45.194:33871): Not found: Tablet not found: 2c62a5ffa9e24703bcb0568cd43588f8
W20260806 22:47:52.859171 7628 leader_election.cc:341] T 2c62a5ffa9e24703bcb0568cd43588f8 P 3c4699a06f354c3db12e2b056bd7ccfb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 970e2afa7c324e03aa24b28063eae728 (127.18.45.195:36453): Not found: Tablet not found: 2c62a5ffa9e24703bcb0568cd43588f8
I20260806 22:47:52.859335 7628 leader_election.cc:302] T 2c62a5ffa9e24703bcb0568cd43588f8 P 3c4699a06f354c3db12e2b056bd7ccfb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c4699a06f354c3db12e2b056bd7ccfb; no voters: 970e2afa7c324e03aa24b28063eae728, f103722ba11f441dbda1143461876f35
I20260806 22:47:52.859457 8208 raft_consensus.cc:2726] T 2c62a5ffa9e24703bcb0568cd43588f8 P 3c4699a06f354c3db12e2b056bd7ccfb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 22:47:52.949573 8209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:52.949174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39797 (local address 127.18.45.195:53955)
0806 22:47:52.949410 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:52.949460 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":149,"threads_started":1}
I20260806 22:47:53.705689 8211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:53.705365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39797 (local address 127.18.45.193:43979)
0806 22:47:53.705558 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:53.705617 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":92,"threads_started":1}
I20260806 22:47:53.782550 8213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:53.782194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39797 (local address 127.18.45.194:39965)
0806 22:47:53.782397 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:53.782477 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":92,"threads_started":1}
I20260806 22:47:53.950745 8217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:53.950367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39797 (local address 127.18.45.195:53007)
0806 22:47:53.950599 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:53.950670 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":133,"threads_started":1}
I20260806 22:47:54.706878 8218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:54.706517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39797 (local address 127.18.45.193:47639)
0806 22:47:54.706745 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:54.706805 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":111,"threads_started":1}
I20260806 22:47:54.783708 8219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:54.783370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39797 (local address 127.18.45.194:57579)
0806 22:47:54.783570 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:54.783631 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":90,"threads_started":1}
I20260806 22:47:54.951898 8220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:54.951530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39797 (local address 127.18.45.195:58833)
0806 22:47:54.951769 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:54.951821 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":119,"threads_started":1}
I20260806 22:47:55.707976 8221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:55.707604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39797 (local address 127.18.45.193:45303)
0806 22:47:55.707807 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:55.707900 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":130,"threads_started":1}
I20260806 22:47:55.784745 8223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:55.784422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39797 (local address 127.18.45.194:50993)
0806 22:47:55.784623 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:55.784671 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":113,"threads_started":1}
I20260806 22:47:55.952953 8224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:55.952578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39797 (local address 127.18.45.195:47703)
0806 22:47:55.952795 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:55.952853 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":113,"threads_started":1}
I20260806 22:47:56.004056 8225 raft_consensus.cc:482] T 2c62a5ffa9e24703bcb0568cd43588f8 P 3c4699a06f354c3db12e2b056bd7ccfb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:47:56.004242 8225 raft_consensus.cc:504] T 2c62a5ffa9e24703bcb0568cd43588f8 P 3c4699a06f354c3db12e2b056bd7ccfb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f103722ba11f441dbda1143461876f35" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 33871 } } peers { permanent_uuid: "970e2afa7c324e03aa24b28063eae728" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 36453 } } peers { permanent_uuid: "3c4699a06f354c3db12e2b056bd7ccfb" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 39517 } }
I20260806 22:47:56.004522 8225 leader_election.cc:288] T 2c62a5ffa9e24703bcb0568cd43588f8 P 3c4699a06f354c3db12e2b056bd7ccfb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f103722ba11f441dbda1143461876f35 (127.18.45.194:33871), 970e2afa7c324e03aa24b28063eae728 (127.18.45.195:36453)
W20260806 22:47:56.005060 7628 leader_election.cc:341] T 2c62a5ffa9e24703bcb0568cd43588f8 P 3c4699a06f354c3db12e2b056bd7ccfb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 970e2afa7c324e03aa24b28063eae728 (127.18.45.195:36453): Not found: Tablet not found: 2c62a5ffa9e24703bcb0568cd43588f8
W20260806 22:47:56.005157 7627 leader_election.cc:341] T 2c62a5ffa9e24703bcb0568cd43588f8 P 3c4699a06f354c3db12e2b056bd7ccfb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f103722ba11f441dbda1143461876f35 (127.18.45.194:33871): Not found: Tablet not found: 2c62a5ffa9e24703bcb0568cd43588f8
I20260806 22:47:56.005208 7627 leader_election.cc:302] T 2c62a5ffa9e24703bcb0568cd43588f8 P 3c4699a06f354c3db12e2b056bd7ccfb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3c4699a06f354c3db12e2b056bd7ccfb; no voters: 970e2afa7c324e03aa24b28063eae728, f103722ba11f441dbda1143461876f35
I20260806 22:47:56.005338 8225 raft_consensus.cc:2726] T 2c62a5ffa9e24703bcb0568cd43588f8 P 3c4699a06f354c3db12e2b056bd7ccfb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 22:47:56.708985 8227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:56.708645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39797 (local address 127.18.45.193:50685)
0806 22:47:56.708819 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:56.708875 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":86,"threads_started":1}
I20260806 22:47:56.785697 8228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:56.785391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39797 (local address 127.18.45.194:51807)
0806 22:47:56.785578 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:56.785622 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":90,"threads_started":1}
I20260806 22:47:56.954119 8229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:56.953754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39797 (local address 127.18.45.195:58051)
0806 22:47:56.953993 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:56.954047 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":84,"threads_started":1}
I20260806 22:47:57.710119 8230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:57.709777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39797 (local address 127.18.45.193:36303)
0806 22:47:57.709996 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:57.710047 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":105,"threads_started":1}
I20260806 22:47:57.786854 8231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:57.786497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39797 (local address 127.18.45.194:54905)
0806 22:47:57.786723 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:57.786778 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":113,"threads_started":1}
I20260806 22:47:57.942905 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 7564
I20260806 22:47:57.955375 8233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:57.954989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39797 (local address 127.18.45.195:43961)
0806 22:47:57.955235 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:57.955292 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:39797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":113,"threads_started":1}
I20260806 22:47:57.958799 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 7792
I20260806 22:47:57.974053 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 7961
2026-08-06T22:47:57Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19204 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:47:58 dist-test-slave-h3ft krb5kdc[8237](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:47:58 dist-test-slave-h3ft krb5kdc[8237](info): set up 2 sockets
Aug 06 22:47:58 dist-test-slave-h3ft krb5kdc[8237](info): commencing operation
krb5kdc: starting...
W20260806 22:47:59.262116 18615 mini_kdc.cc:121] Time spent starting KDC: real 1.260s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Aug 06 22:47:59 dist-test-slave-h3ft krb5kdc[8237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056479, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:47:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:47:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.45.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab.
Entry for principal kudu/127.18.45.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab.
I20260806 22:47:59.402469 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:32823
--webserver_interface=127.18.45.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:41985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.45.254:32823
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:47:59.500828 8253 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:59.501117 8253 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:59.501158 8253 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:59.502442 8253 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 22:47:59.502512 8253 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 22:47:59.502537 8253 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 22:47:59.502569 8253 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:47:59.502599 8253 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:59.502627 8253 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:59.502653 8253 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:47:59.502679 8253 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:47:59.504459 8253 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:41985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.18.45.254:32823
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:32823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.45.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8253
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:47:59.504788 8253 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:59.505088 8253 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 22:47:59.511457 8253 server_base.cc:1034] running on GCE node
W20260806 22:47:59.511436 8260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:59.511436 8258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:59.511417 8263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:59.512022 8253 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:59.512413 8253 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:59.513617 8253 hybrid_clock.cc:648] HybridClock initialized: now 1786056479513591 us; error 38 us; skew 500 ppm
Aug 06 22:47:59 dist-test-slave-h3ft krb5kdc[8237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056479, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:59.516778 8253 init.cc:377] Logged in from keytab as kudu/127.18.45.254@KRBTEST.COM (short username kudu)
I20260806 22:47:59.517324 8253 webserver.cc:460] Webserver started at http://127.18.45.254:41899/ using document root <none> and password file <none>
I20260806 22:47:59.517585 8253 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:59.517632 8253 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:59.517737 8253 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:47:59.519064 8253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/master-0/data/instance:
uuid: "2817a20f1830402dbc70359306e75419"
format_stamp: "Formatted at 2026-08-06 22:47:59 on dist-test-slave-h3ft"
server_key: "2e9805f58a1ffd833fa20b15e90cddc4"
server_key_iv: "95356632211c9ff1b9aac382ea71a14c"
server_key_version: "encryptionkey@0"
I20260806 22:47:59.519560 8253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/master-0/wal/instance:
uuid: "2817a20f1830402dbc70359306e75419"
format_stamp: "Formatted at 2026-08-06 22:47:59 on dist-test-slave-h3ft"
server_key: "2e9805f58a1ffd833fa20b15e90cddc4"
server_key_iv: "95356632211c9ff1b9aac382ea71a14c"
server_key_version: "encryptionkey@0"
I20260806 22:47:59.521029 8253 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:47:59.521852 8270 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:59.522061 8253 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:47:59.522161 8253 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/master-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/master-0/wal
uuid: "2817a20f1830402dbc70359306e75419"
format_stamp: "Formatted at 2026-08-06 22:47:59 on dist-test-slave-h3ft"
server_key: "2e9805f58a1ffd833fa20b15e90cddc4"
server_key_iv: "95356632211c9ff1b9aac382ea71a14c"
server_key_version: "encryptionkey@0"
I20260806 22:47:59.522220 8253 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:59.557031 8253 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:59.559962 8253 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:59.560148 8253 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:59.565620 8253 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.254:32823
I20260806 22:47:59.565682 8334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.254:32823 every 8 connection(s)
I20260806 22:47:59.566118 8253 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
I20260806 22:47:59.567384 8335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:47:59.570360 8335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419: Bootstrap starting.
I20260806 22:47:59.570461 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 8253
I20260806 22:47:59.570595 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/master-0/wal/instance
I20260806 22:47:59.570852 18615 external_mini_cluster.cc:1428] Setting key 04b22fdfa035d7a91588213fc326f7ee
I20260806 22:47:59.571365 8335 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419: Neither blocks nor log segments found. Creating new log.
I20260806 22:47:59.571784 8335 log.cc:824] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419: Log is configured to *not* fsync() on all Append() calls
I20260806 22:47:59.572647 8335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419: No bootstrap required, opened a new log
I20260806 22:47:59.574846 8335 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2817a20f1830402dbc70359306e75419" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 32823 } }
I20260806 22:47:59.575062 8335 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:47:59.575117 8335 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2817a20f1830402dbc70359306e75419, State: Initialized, Role: FOLLOWER
I20260806 22:47:59.575271 8335 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2817a20f1830402dbc70359306e75419" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 32823 } }
I20260806 22:47:59.575363 8335 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:47:59.575409 8335 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:47:59.575464 8335 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:47:59.576093 8335 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2817a20f1830402dbc70359306e75419" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 32823 } }
I20260806 22:47:59.576226 8335 leader_election.cc:302] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2817a20f1830402dbc70359306e75419; no voters:
I20260806 22:47:59.576442 8335 leader_election.cc:288] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:47:59.576488 8340 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:47:59.576615 8340 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419 [term 1 LEADER]: Becoming Leader. State: Replica: 2817a20f1830402dbc70359306e75419, State: Running, Role: LEADER
I20260806 22:47:59.576792 8335 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:47:59.576785 8340 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2817a20f1830402dbc70359306e75419" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 32823 } }
I20260806 22:47:59.577286 8341 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2817a20f1830402dbc70359306e75419. Latest consensus state: current_term: 1 leader_uuid: "2817a20f1830402dbc70359306e75419" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2817a20f1830402dbc70359306e75419" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 32823 } } }
I20260806 22:47:59.577395 8341 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419 [sys.catalog]: This master's current role is: LEADER
I20260806 22:47:59.578377 8342 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2817a20f1830402dbc70359306e75419" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2817a20f1830402dbc70359306e75419" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 32823 } } }
I20260806 22:47:59.578459 8342 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419 [sys.catalog]: This master's current role is: LEADER
Aug 06 22:47:59 dist-test-slave-h3ft krb5kdc[8237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056479, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
W20260806 22:47:59.579104 8357 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:47:59.579172 8357 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:47:59.580437 8347 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:47:59.580667 8347 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:47:59.583072 8347 catalog_manager.cc:1347] Generated new cluster ID: 1b4274433ca54a8183f72fe367b61dba
I20260806 22:47:59.583159 8347 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:47:59.584169 8338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:59.571542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40900 (local address 127.18.45.254:32823)
0806 22:47:59.571779 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:59.571790 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:59.571850 (+ 60us) server_negotiation.cc:407] Connection header received
0806 22:47:59.572498 (+ 648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:59.572507 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:59.572584 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:59.572759 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:59.573491 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:59.574490 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:59.575565 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:59.576005 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:59.576141 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:59.579573 (+ 3432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:59.579578 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:59.579579 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:59.579593 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:59.582047 (+ 2454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:59.582586 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:59.582591 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:59.582592 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:59.582733 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:59.582990 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:59.582991 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:59.582992 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:59.583405 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:59.583600 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:59.583717 (+ 117us) server_negotiation.cc:299] Negotiation successful
0806 22:47:59.583890 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":100,"threads_started":1}
I20260806 22:47:59.592731 8347 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:47:59.593389 8347 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:47:59.597493 8347 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419: Generated new TSK 0
I20260806 22:47:59.597744 8347 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.45.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab.
Entry for principal kudu/127.18.45.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab.
I20260806 22:47:59.634477 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.193:0
--local_ip_for_outbound_sockets=127.18.45.193
--webserver_interface=127.18.45.193
--webserver_port=0
--tserver_master_addrs=127.18.45.254:32823
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab
--principal=kudu/127.18.45.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:41985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:47:59.732705 8369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:59.732990 8369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:59.733017 8369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:59.734344 8369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:59.734400 8369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:59.734432 8369 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.193
I20260806 22:47:59.736174 8369 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:41985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab
--principal=kudu/127.18.45.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.45.193
--webserver_port=0
--tserver_master_addrs=127.18.45.254:32823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8369
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.193
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:47:59.736513 8369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:59.736822 8369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:59.743331 8375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:59.743332 8376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:59.743402 8378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:59.743681 8369 server_base.cc:1034] running on GCE node
I20260806 22:47:59.743992 8369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:59.744385 8369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:59.745548 8369 hybrid_clock.cc:648] HybridClock initialized: now 1786056479745524 us; error 34 us; skew 500 ppm
Aug 06 22:47:59 dist-test-slave-h3ft krb5kdc[8237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056479, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:59.748950 8369 init.cc:377] Logged in from keytab as kudu/127.18.45.193@KRBTEST.COM (short username kudu)
I20260806 22:47:59.749570 8369 webserver.cc:460] Webserver started at http://127.18.45.193:38949/ using document root <none> and password file <none>
I20260806 22:47:59.749858 8369 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:59.749940 8369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:59.750131 8369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:47:59.751636 8369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-0/data/instance:
uuid: "e70eeb6d81fd488b9c80b9c6de85eccc"
format_stamp: "Formatted at 2026-08-06 22:47:59 on dist-test-slave-h3ft"
server_key: "cd42bc14687e788551e73e3f92592154"
server_key_iv: "2e1e4ec8c02c735da89f96ce03bed992"
server_key_version: "encryptionkey@0"
I20260806 22:47:59.752095 8369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-0/wal/instance:
uuid: "e70eeb6d81fd488b9c80b9c6de85eccc"
format_stamp: "Formatted at 2026-08-06 22:47:59 on dist-test-slave-h3ft"
server_key: "cd42bc14687e788551e73e3f92592154"
server_key_iv: "2e1e4ec8c02c735da89f96ce03bed992"
server_key_version: "encryptionkey@0"
I20260806 22:47:59.753829 8369 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:47:59.754869 8390 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:59.755128 8369 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:47:59.755268 8369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-0/wal
uuid: "e70eeb6d81fd488b9c80b9c6de85eccc"
format_stamp: "Formatted at 2026-08-06 22:47:59 on dist-test-slave-h3ft"
server_key: "cd42bc14687e788551e73e3f92592154"
server_key_iv: "2e1e4ec8c02c735da89f96ce03bed992"
server_key_version: "encryptionkey@0"
I20260806 22:47:59.755365 8369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:59.768939 8369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:59.772035 8369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:59.772230 8369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:59.772544 8369 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:47:59.772962 8369 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:47:59.773074 8369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:59.773137 8369 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:47:59.773173 8369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:59.781457 8369 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.193:41625
I20260806 22:47:59.781567 8542 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.193:41625 every 8 connection(s)
I20260806 22:47:59.781996 8369 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
Aug 06 22:47:59 dist-test-slave-h3ft krb5kdc[8237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056479, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.193@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:47:59.791989 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 8369
I20260806 22:47:59.792162 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-0/wal/instance
I20260806 22:47:59.792443 18615 external_mini_cluster.cc:1428] Setting key e768963e425452af7bcd1415b8730b7e
I20260806 22:47:59.793401 8338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:59.783089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:38323 (local address 127.18.45.254:32823)
0806 22:47:59.783174 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:59.783177 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:59.783385 (+ 208us) server_negotiation.cc:407] Connection header received
0806 22:47:59.783986 (+ 601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:59.783987 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:59.784000 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:59.784075 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:59.785256 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:59.785888 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:59.787086 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:59.787349 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:59.787413 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:59.789937 (+ 2524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:59.789939 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:59.789939 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:59.789947 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:59.791528 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:59.792086 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:59.792087 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:59.792089 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:59.792166 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:59.792558 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:59.792559 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:59.792560 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:59.792716 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:59.792797 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:59.793152 (+ 355us) server_negotiation.cc:299] Negotiation successful
0806 22:47:59.793282 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 22:47:59.793609 8548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:59.782955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:32823 (local address 127.18.45.193:38323)
0806 22:47:59.783214 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:59.783267 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:59.783885 (+ 618us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:59.784145 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:59.784149 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:59.784876 (+ 727us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:59.785137 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:59.785143 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:59.786016 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:59.786017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:59.786917 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:59.786919 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:59.787432 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:59.787433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:59.787559 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:59.788199 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:59.788208 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:59.789840 (+ 1632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:59.791597 (+ 1757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:59.791604 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:59.791606 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:59.791944 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:59.792289 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:59.792290 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:59.792290 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:59.792446 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:59.792793 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:59.792818 (+ 25us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:59.793009 (+ 191us) client_negotiation.cc:769] Sending connection context
0806 22:47:59.793107 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 22:47:59.793322 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":163,"threads_started":1}
I20260806 22:47:59.794361 8543 heartbeater.cc:347] Connected to a master server at 127.18.45.254:32823
I20260806 22:47:59.794488 8543 heartbeater.cc:464] Registering TS with master...
I20260806 22:47:59.794739 8543 heartbeater.cc:511] Master 127.18.45.254:32823 requested a full tablet report, sending...
I20260806 22:47:59.795588 8291 ts_manager.cc:194] Registered new tserver with Master: e70eeb6d81fd488b9c80b9c6de85eccc (127.18.45.193:41625)
I20260806 22:47:59.796429 8291 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.45.193@KRBTEST.COM'} at 127.18.45.193:38323
WARNING: no policy specified for kudu/127.18.45.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab.
Entry for principal kudu/127.18.45.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab.
I20260806 22:47:59.826530 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.194:0
--local_ip_for_outbound_sockets=127.18.45.194
--webserver_interface=127.18.45.194
--webserver_port=0
--tserver_master_addrs=127.18.45.254:32823
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab
--principal=kudu/127.18.45.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:41985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:47:59.927505 8551 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:47:59.927848 8551 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:47:59.927881 8551 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:47:59.929334 8551 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:47:59.929382 8551 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:47:59.929440 8551 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.194
I20260806 22:47:59.931258 8551 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:41985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab
--principal=kudu/127.18.45.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.45.194
--webserver_port=0
--tserver_master_addrs=127.18.45.254:32823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8551
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.194
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:47:59.931586 8551 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:47:59.931892 8551 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:47:59.938349 8560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:59.938376 8556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:47:59.938357 8557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:47:59.938699 8551 server_base.cc:1034] running on GCE node
I20260806 22:47:59.938956 8551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:47:59.939327 8551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:47:59.940510 8551 hybrid_clock.cc:648] HybridClock initialized: now 1786056479940487 us; error 46 us; skew 500 ppm
Aug 06 22:47:59 dist-test-slave-h3ft krb5kdc[8237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056479, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:47:59.944087 8551 init.cc:377] Logged in from keytab as kudu/127.18.45.194@KRBTEST.COM (short username kudu)
I20260806 22:47:59.944646 8551 webserver.cc:460] Webserver started at http://127.18.45.194:38907/ using document root <none> and password file <none>
I20260806 22:47:59.944933 8551 fs_manager.cc:359] Metadata directory not provided
I20260806 22:47:59.945011 8551 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:47:59.945183 8551 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:47:59.946472 8551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-1/data/instance:
uuid: "4756eb5819a543f4bbf440e26deb8284"
format_stamp: "Formatted at 2026-08-06 22:47:59 on dist-test-slave-h3ft"
server_key: "0889662080420627d084d3b1538d3ef9"
server_key_iv: "8851ad8f354a9ce5e6d2bd0c3067b1b7"
server_key_version: "encryptionkey@0"
I20260806 22:47:59.946903 8551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-1/wal/instance:
uuid: "4756eb5819a543f4bbf440e26deb8284"
format_stamp: "Formatted at 2026-08-06 22:47:59 on dist-test-slave-h3ft"
server_key: "0889662080420627d084d3b1538d3ef9"
server_key_iv: "8851ad8f354a9ce5e6d2bd0c3067b1b7"
server_key_version: "encryptionkey@0"
I20260806 22:47:59.948563 8551 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:47:59.949504 8571 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:59.949826 8551 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:47:59.949997 8551 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-1/wal
uuid: "4756eb5819a543f4bbf440e26deb8284"
format_stamp: "Formatted at 2026-08-06 22:47:59 on dist-test-slave-h3ft"
server_key: "0889662080420627d084d3b1538d3ef9"
server_key_iv: "8851ad8f354a9ce5e6d2bd0c3067b1b7"
server_key_version: "encryptionkey@0"
I20260806 22:47:59.950104 8551 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:47:59.968261 8551 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:47:59.971621 8551 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:47:59.971859 8551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:47:59.972220 8551 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:47:59.972708 8551 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:47:59.972792 8551 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:59.972865 8551 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:47:59.972900 8551 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:47:59.981848 8551 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.194:34433
I20260806 22:47:59.981962 8720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.194:34433 every 8 connection(s)
I20260806 22:47:59.982404 8551 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
I20260806 22:47:59.984018 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 8551
I20260806 22:47:59.984148 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-1/wal/instance
I20260806 22:47:59.984360 18615 external_mini_cluster.cc:1428] Setting key 22a34c0aaa682c0dfaaef99b79a714d3
Aug 06 22:47:59 dist-test-slave-h3ft krb5kdc[8237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056479, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.194@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:47:59.996049 8338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:59.983473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:47985 (local address 127.18.45.254:32823)
0806 22:47:59.983572 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0806 22:47:59.983575 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:47:59.983825 (+ 250us) server_negotiation.cc:407] Connection header received
0806 22:47:59.984688 (+ 863us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:47:59.984690 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:47:59.984703 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:47:59.984805 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:47:59.986283 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:59.987103 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:59.988370 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:47:59.988707 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:47:59.988790 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:59.991887 (+ 3097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:47:59.991889 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:47:59.991890 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:47:59.991901 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:47:59.994105 (+ 2204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:59.994635 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:59.994638 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:59.994638 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:59.994721 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:47:59.995162 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:47:59.995163 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:47:59.995163 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:47:59.995358 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:47:59.995447 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 22:47:59.995779 (+ 332us) server_negotiation.cc:299] Negotiation successful
0806 22:47:59.995899 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 22:47:59.996169 8724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:47:59.983396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:32823 (local address 127.18.45.194:47985)
0806 22:47:59.983639 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0806 22:47:59.983690 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 22:47:59.984529 (+ 839us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:47:59.984950 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:47:59.984955 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:47:59.985886 (+ 931us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:47:59.986154 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:59.986160 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:59.987229 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:59.987230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:59.988274 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:47:59.988277 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:47:59.988814 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:47:59.988815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:47:59.989008 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:47:59.989703 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:47:59.989715 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:47:59.991774 (+ 2059us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:47:59.994224 (+ 2450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:59.994233 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:59.994236 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:59.994542 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:59.994827 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:47:59.994828 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:47:59.994829 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:47:59.994962 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:47:59.995447 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:47:59.995454 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:47:59.995621 (+ 167us) client_negotiation.cc:769] Sending connection context
0806 22:47:59.995720 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 22:47:59.995889 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":132,"threads_started":1}
I20260806 22:47:59.996886 8721 heartbeater.cc:347] Connected to a master server at 127.18.45.254:32823
I20260806 22:47:59.997042 8721 heartbeater.cc:464] Registering TS with master...
I20260806 22:47:59.997282 8721 heartbeater.cc:511] Master 127.18.45.254:32823 requested a full tablet report, sending...
I20260806 22:47:59.997816 8291 ts_manager.cc:194] Registered new tserver with Master: 4756eb5819a543f4bbf440e26deb8284 (127.18.45.194:34433)
WARNING: no policy specified for kudu/127.18.45.195@KRBTEST.COM; defaulting to no policy
I20260806 22:47:59.998464 8291 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.45.194@KRBTEST.COM'} at 127.18.45.194:47985
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab.
Entry for principal kudu/127.18.45.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab.
I20260806 22:48:00.023082 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.195:0
--local_ip_for_outbound_sockets=127.18.45.195
--webserver_interface=127.18.45.195
--webserver_port=0
--tserver_master_addrs=127.18.45.254:32823
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab
--principal=kudu/127.18.45.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:41985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:48:00.125167 8728 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:48:00.125437 8728 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:48:00.125492 8728 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:48:00.126740 8728 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:48:00.126788 8728 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:48:00.126816 8728 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.195
I20260806 22:48:00.128472 8728 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:41985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab
--principal=kudu/127.18.45.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.45.195
--webserver_port=0
--tserver_master_addrs=127.18.45.254:32823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8728
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.195
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:48:00.128719 8728 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:48:00.128960 8728 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:48:00.135300 8736 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:48:00.135308 8735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:48:00.135437 8728 server_base.cc:1034] running on GCE node
W20260806 22:48:00.135440 8739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:48:00.135829 8728 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:48:00.136132 8728 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:48:00.137308 8728 hybrid_clock.cc:648] HybridClock initialized: now 1786056480137302 us; error 49 us; skew 500 ppm
Aug 06 22:48:00 dist-test-slave-h3ft krb5kdc[8237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056480, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:48:00.140877 8728 init.cc:377] Logged in from keytab as kudu/127.18.45.195@KRBTEST.COM (short username kudu)
I20260806 22:48:00.141484 8728 webserver.cc:460] Webserver started at http://127.18.45.195:42291/ using document root <none> and password file <none>
I20260806 22:48:00.141708 8728 fs_manager.cc:359] Metadata directory not provided
I20260806 22:48:00.141752 8728 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:48:00.141858 8728 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:48:00.143143 8728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-2/data/instance:
uuid: "53a97834ac9e48e8b3c5ad90490c9d25"
format_stamp: "Formatted at 2026-08-06 22:48:00 on dist-test-slave-h3ft"
server_key: "8b3fc2654f66064554342d0522709aaf"
server_key_iv: "9681cd6fa5631a2d288a7784fd80f65b"
server_key_version: "encryptionkey@0"
I20260806 22:48:00.143532 8728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-2/wal/instance:
uuid: "53a97834ac9e48e8b3c5ad90490c9d25"
format_stamp: "Formatted at 2026-08-06 22:48:00 on dist-test-slave-h3ft"
server_key: "8b3fc2654f66064554342d0522709aaf"
server_key_iv: "9681cd6fa5631a2d288a7784fd80f65b"
server_key_version: "encryptionkey@0"
I20260806 22:48:00.145138 8728 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 22:48:00.145949 8749 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:48:00.146160 8728 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:48:00.146248 8728 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-2/wal
uuid: "53a97834ac9e48e8b3c5ad90490c9d25"
format_stamp: "Formatted at 2026-08-06 22:48:00 on dist-test-slave-h3ft"
server_key: "8b3fc2654f66064554342d0522709aaf"
server_key_iv: "9681cd6fa5631a2d288a7784fd80f65b"
server_key_version: "encryptionkey@0"
I20260806 22:48:00.146313 8728 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:48:00.170362 8728 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:48:00.173223 8728 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:48:00.173420 8728 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:48:00.173771 8728 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:48:00.174218 8728 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:48:00.174258 8728 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:48:00.174309 8728 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:48:00.174337 8728 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:48:00.182516 8728 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.195:45699
I20260806 22:48:00.182567 8905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.195:45699 every 8 connection(s)
I20260806 22:48:00.182987 8728 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
Aug 06 22:48:00 dist-test-slave-h3ft krb5kdc[8237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056480, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.195@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:48:00.192313 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 8728
I20260806 22:48:00.192469 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-2/wal/instance
I20260806 22:48:00.192718 18615 external_mini_cluster.cc:1428] Setting key a115e84f654c2c6f7e1e072f085ab085
I20260806 22:48:00.195746 8338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:00.183985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.195:35101 (local address 127.18.45.254:32823)
0806 22:48:00.184075 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0806 22:48:00.184078 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:48:00.184354 (+ 276us) server_negotiation.cc:407] Connection header received
0806 22:48:00.185019 (+ 665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:48:00.185020 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:48:00.185036 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:48:00.185113 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:48:00.186419 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:00.187188 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:00.188488 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:00.188718 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:00.188796 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:00.191681 (+ 2885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:48:00.191683 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:48:00.191683 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:48:00.191693 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:48:00.194064 (+ 2371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:48:00.194601 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:48:00.194602 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:48:00.194603 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:48:00.194682 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:48:00.194951 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:48:00.194953 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:48:00.194953 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:48:00.195176 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:48:00.195243 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 22:48:00.195562 (+ 319us) server_negotiation.cc:299] Negotiation successful
0806 22:48:00.195662 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 22:48:00.196033 8910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:00.183985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:32823 (local address 127.18.45.195:35101)
0806 22:48:00.184201 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:00.184248 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 22:48:00.184880 (+ 632us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:48:00.185254 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:48:00.185259 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:48:00.186001 (+ 742us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:48:00.186263 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:48:00.186269 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:48:00.187300 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:48:00.187302 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:48:00.188360 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:48:00.188364 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:48:00.188829 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:48:00.188831 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:48:00.188987 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:00.189696 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:48:00.189707 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:48:00.191550 (+ 1843us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:48:00.194214 (+ 2664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:48:00.194224 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:48:00.194226 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:48:00.194514 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:48:00.194760 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:48:00.194760 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:48:00.194761 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:48:00.194889 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:48:00.195242 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:48:00.195249 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:48:00.195412 (+ 163us) client_negotiation.cc:769] Sending connection context
0806 22:48:00.195517 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 22:48:00.195688 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":113,"threads_started":1}
I20260806 22:48:00.196794 8908 heartbeater.cc:347] Connected to a master server at 127.18.45.254:32823
I20260806 22:48:00.196931 8908 heartbeater.cc:464] Registering TS with master...
I20260806 22:48:00.197261 8908 heartbeater.cc:511] Master 127.18.45.254:32823 requested a full tablet report, sending...
I20260806 22:48:00.197844 8291 ts_manager.cc:194] Registered new tserver with Master: 53a97834ac9e48e8b3c5ad90490c9d25 (127.18.45.195:45699)
I20260806 22:48:00.198342 8291 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.45.195@KRBTEST.COM'} at 127.18.45.195:35101
I20260806 22:48:00.205437 18615 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:48:00.215662 8338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:00.207506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40904 (local address 127.18.45.254:32823)
0806 22:48:00.207596 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0806 22:48:00.207598 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:48:00.207727 (+ 129us) server_negotiation.cc:407] Connection header received
0806 22:48:00.207761 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:48:00.207762 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:48:00.207774 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:48:00.207835 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:48:00.208591 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:00.209348 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:00.210357 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:00.210591 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:00.210667 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:00.212423 (+ 1756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:48:00.212425 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:48:00.212425 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:48:00.212433 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:48:00.214188 (+ 1755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:48:00.214631 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:48:00.214632 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:48:00.214633 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:48:00.214694 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:48:00.214902 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:48:00.214902 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:48:00.214903 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:48:00.215222 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:48:00.215320 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 22:48:00.215499 (+ 179us) server_negotiation.cc:299] Negotiation successful
0806 22:48:00.215575 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 22:48:00.218051 8291 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40904:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:48:00.220484 8291 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:48:00.229017 8930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:00.223742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.193:41625 (local address 127.0.0.1:44336)
0806 22:48:00.224659 (+ 917us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:00.224669 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:48:00.224750 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:48:00.224959 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:48:00.224960 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:48:00.224962 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:48:00.225022 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:48:00.225024 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:48:00.226449 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:48:00.226449 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:48:00.227709 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:48:00.227712 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:48:00.228724 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:48:00.228724 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:48:00.228820 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:00.228823 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:48:00.228869 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 22:48:00.228923 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":892,"mutex_wait_us":36,"thread_start_us":80,"threads_started":1}
I20260806 22:48:00.229159 8934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:00.224388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39686 (local address 127.18.45.194:34433)
0806 22:48:00.224616 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0806 22:48:00.224622 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:48:00.224634 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:48:00.224693 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:48:00.224696 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:48:00.224716 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:48:00.224810 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:48:00.225090 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:00.226331 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:00.227702 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:00.228509 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:00.228592 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:00.228659 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 22:48:00.228976 (+ 317us) server_negotiation.cc:299] Negotiation successful
0806 22:48:00.229048 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":146,"threads_started":1}
I20260806 22:48:00.229166 8932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:00.224101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47016 (local address 127.18.45.195:45699)
0806 22:48:00.224302 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0806 22:48:00.224308 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:48:00.224919 (+ 611us) server_negotiation.cc:407] Connection header received
0806 22:48:00.225055 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:48:00.225058 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:48:00.225078 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:48:00.225136 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:48:00.225327 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:00.226119 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:00.227853 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:00.228455 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:00.228555 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:00.228633 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 22:48:00.228976 (+ 343us) server_negotiation.cc:299] Negotiation successful
0806 22:48:00.229035 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":109,"threads_started":1}
I20260806 22:48:00.229377 8928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:00.223744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.195:45699 (local address 127.0.0.1:47016)
0806 22:48:00.224847 (+ 1103us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:00.224856 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:48:00.224950 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:48:00.225222 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:48:00.225223 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:48:00.225224 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:48:00.225255 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:48:00.225256 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:48:00.226244 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:48:00.226246 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:48:00.227744 (+ 1498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:48:00.227745 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:48:00.228554 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:48:00.228555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:48:00.228642 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:00.228672 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 22:48:00.228776 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 22:48:00.228857 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1082,"mutex_wait_us":41,"thread_start_us":132,"threads_started":1}
I20260806 22:48:00.229490 8933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:00.224177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44336 (local address 127.18.45.193:41625)
0806 22:48:00.224429 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0806 22:48:00.224434 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:48:00.224739 (+ 305us) server_negotiation.cc:407] Connection header received
0806 22:48:00.224839 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:48:00.224842 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:48:00.224860 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:48:00.224992 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:48:00.225102 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:00.226364 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:00.227817 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:00.228579 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:00.228642 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:00.228693 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 22:48:00.229284 (+ 591us) server_negotiation.cc:299] Negotiation successful
0806 22:48:00.229374 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":109,"threads_started":1}
I20260806 22:48:00.229564 8926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:00.223741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.194:34433 (local address 127.0.0.1:39686)
0806 22:48:00.223967 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:00.224015 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 22:48:00.224116 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:48:00.224829 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:48:00.224832 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:48:00.224836 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:48:00.224938 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:48:00.224940 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:48:00.226655 (+ 1715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:48:00.226656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:48:00.227594 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:48:00.227597 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:48:00.228698 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:48:00.228699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:48:00.228805 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:00.228810 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:48:00.228865 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 22:48:00.228922 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"mutex_wait_us":3,"thread_start_us":152,"threads_started":1}
I20260806 22:48:00.230400 8455 tablet_service.cc:1467] Processing CreateTablet for tablet ee1b323fca4b4bdabf8a98ccea8e57f0 (DEFAULT_TABLE table=test-table [id=9d187abacfb1498b9bd1864b7f5fb0d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:48:00.230417 8820 tablet_service.cc:1467] Processing CreateTablet for tablet ee1b323fca4b4bdabf8a98ccea8e57f0 (DEFAULT_TABLE table=test-table [id=9d187abacfb1498b9bd1864b7f5fb0d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:48:00.230417 8637 tablet_service.cc:1467] Processing CreateTablet for tablet ee1b323fca4b4bdabf8a98ccea8e57f0 (DEFAULT_TABLE table=test-table [id=9d187abacfb1498b9bd1864b7f5fb0d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:48:00.230826 8455 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee1b323fca4b4bdabf8a98ccea8e57f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:48:00.230829 8637 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee1b323fca4b4bdabf8a98ccea8e57f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:48:00.230829 8820 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee1b323fca4b4bdabf8a98ccea8e57f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:48:00.235554 8936 tablet_bootstrap.cc:492] T ee1b323fca4b4bdabf8a98ccea8e57f0 P e70eeb6d81fd488b9c80b9c6de85eccc: Bootstrap starting.
I20260806 22:48:00.235718 8935 tablet_bootstrap.cc:492] T ee1b323fca4b4bdabf8a98ccea8e57f0 P 53a97834ac9e48e8b3c5ad90490c9d25: Bootstrap starting.
I20260806 22:48:00.236497 8937 tablet_bootstrap.cc:492] T ee1b323fca4b4bdabf8a98ccea8e57f0 P 4756eb5819a543f4bbf440e26deb8284: Bootstrap starting.
I20260806 22:48:00.236580 8935 tablet_bootstrap.cc:654] T ee1b323fca4b4bdabf8a98ccea8e57f0 P 53a97834ac9e48e8b3c5ad90490c9d25: Neither blocks nor log segments found. Creating new log.
I20260806 22:48:00.236580 8936 tablet_bootstrap.cc:654] T ee1b323fca4b4bdabf8a98ccea8e57f0 P e70eeb6d81fd488b9c80b9c6de85eccc: Neither blocks nor log segments found. Creating new log.
I20260806 22:48:00.237299 8935 log.cc:824] T ee1b323fca4b4bdabf8a98ccea8e57f0 P 53a97834ac9e48e8b3c5ad90490c9d25: Log is configured to *not* fsync() on all Append() calls
I20260806 22:48:00.237468 8936 log.cc:824] T ee1b323fca4b4bdabf8a98ccea8e57f0 P e70eeb6d81fd488b9c80b9c6de85eccc: Log is configured to *not* fsync() on all Append() calls
I20260806 22:48:00.237624 8937 tablet_bootstrap.cc:654] T ee1b323fca4b4bdabf8a98ccea8e57f0 P 4756eb5819a543f4bbf440e26deb8284: Neither blocks nor log segments found. Creating new log.
I20260806 22:48:00.238317 8935 tablet_bootstrap.cc:492] T ee1b323fca4b4bdabf8a98ccea8e57f0 P 53a97834ac9e48e8b3c5ad90490c9d25: No bootstrap required, opened a new log
I20260806 22:48:00.238408 8935 ts_tablet_manager.cc:1397] T ee1b323fca4b4bdabf8a98ccea8e57f0 P 53a97834ac9e48e8b3c5ad90490c9d25: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:48:00.238452 8937 log.cc:824] T ee1b323fca4b4bdabf8a98ccea8e57f0 P 4756eb5819a543f4bbf440e26deb8284: Log is configured to *not* fsync() on all Append() calls
I20260806 22:48:00.238745 8936 tablet_bootstrap.cc:492] T ee1b323fca4b4bdabf8a98ccea8e57f0 P e70eeb6d81fd488b9c80b9c6de85eccc: No bootstrap required, opened a new log
I20260806 22:48:00.238826 8936 ts_tablet_manager.cc:1397] T ee1b323fca4b4bdabf8a98ccea8e57f0 P e70eeb6d81fd488b9c80b9c6de85eccc: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:48:00.239375 8937 tablet_bootstrap.cc:492] T ee1b323fca4b4bdabf8a98ccea8e57f0 P 4756eb5819a543f4bbf440e26deb8284: No bootstrap required, opened a new log
I20260806 22:48:00.239477 8937 ts_tablet_manager.cc:1397] T ee1b323fca4b4bdabf8a98ccea8e57f0 P 4756eb5819a543f4bbf440e26deb8284: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:48:00.240773 8935 raft_consensus.cc:348] T ee1b323fca4b4bdabf8a98ccea8e57f0 P 53a97834ac9e48e8b3c5ad90490c9d25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4756eb5819a543f4bbf440e26deb8284" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 34433 } } peers { permanent_uuid: "e70eeb6d81fd488b9c80b9c6de85eccc" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 41625 } } peers { permanent_uuid: "53a97834ac9e48e8b3c5ad90490c9d25" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 45699 } }
I20260806 22:48:00.241010 8935 raft_consensus.cc:374] T ee1b323fca4b4bdabf8a98ccea8e57f0 P 53a97834ac9e48e8b3c5ad90490c9d25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:48:00.241065 8935 raft_consensus.cc:729] T ee1b323fca4b4bdabf8a98ccea8e57f0 P 53a97834ac9e48e8b3c5ad90490c9d25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53a97834ac9e48e8b3c5ad90490c9d25, State: Initialized, Role: FOLLOWER
I20260806 22:48:00.241111 8936 raft_consensus.cc:348] T ee1b323fca4b4bdabf8a98ccea8e57f0 P e70eeb6d81fd488b9c80b9c6de85eccc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4756eb5819a543f4bbf440e26deb8284" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 34433 } } peers { permanent_uuid: "e70eeb6d81fd488b9c80b9c6de85eccc" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 41625 } } peers { permanent_uuid: "53a97834ac9e48e8b3c5ad90490c9d25" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 45699 } }
I20260806 22:48:00.241206 8935 consensus_queue.cc:260] T ee1b323fca4b4bdabf8a98ccea8e57f0 P 53a97834ac9e48e8b3c5ad90490c9d25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4756eb5819a543f4bbf440e26deb8284" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 34433 } } peers { permanent_uuid: "e70eeb6d81fd488b9c80b9c6de85eccc" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 41625 } } peers { permanent_uuid: "53a97834ac9e48e8b3c5ad90490c9d25" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 45699 } }
I20260806 22:48:00.241304 8936 raft_consensus.cc:374] T ee1b323fca4b4bdabf8a98ccea8e57f0 P e70eeb6d81fd488b9c80b9c6de85eccc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:48:00.241343 8936 raft_consensus.cc:729] T ee1b323fca4b4bdabf8a98ccea8e57f0 P e70eeb6d81fd488b9c80b9c6de85eccc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e70eeb6d81fd488b9c80b9c6de85eccc, State: Initialized, Role: FOLLOWER
I20260806 22:48:00.241340 8937 raft_consensus.cc:348] T ee1b323fca4b4bdabf8a98ccea8e57f0 P 4756eb5819a543f4bbf440e26deb8284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4756eb5819a543f4bbf440e26deb8284" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 34433 } } peers { permanent_uuid: "e70eeb6d81fd488b9c80b9c6de85eccc" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 41625 } } peers { permanent_uuid: "53a97834ac9e48e8b3c5ad90490c9d25" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 45699 } }
I20260806 22:48:00.241496 8936 consensus_queue.cc:260] T ee1b323fca4b4bdabf8a98ccea8e57f0 P e70eeb6d81fd488b9c80b9c6de85eccc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4756eb5819a543f4bbf440e26deb8284" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 34433 } } peers { permanent_uuid: "e70eeb6d81fd488b9c80b9c6de85eccc" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 41625 } } peers { permanent_uuid: "53a97834ac9e48e8b3c5ad90490c9d25" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 45699 } }
I20260806 22:48:00.241560 8935 ts_tablet_manager.cc:1428] T ee1b323fca4b4bdabf8a98ccea8e57f0 P 53a97834ac9e48e8b3c5ad90490c9d25: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260806 22:48:00.241560 8937 raft_consensus.cc:374] T ee1b323fca4b4bdabf8a98ccea8e57f0 P 4756eb5819a543f4bbf440e26deb8284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:48:00.241608 8937 raft_consensus.cc:729] T ee1b323fca4b4bdabf8a98ccea8e57f0 P 4756eb5819a543f4bbf440e26deb8284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4756eb5819a543f4bbf440e26deb8284, State: Initialized, Role: FOLLOWER
I20260806 22:48:00.241720 8908 heartbeater.cc:503] Master 127.18.45.254:32823 was elected leader, sending a full tablet report...
I20260806 22:48:00.241757 8937 consensus_queue.cc:260] T ee1b323fca4b4bdabf8a98ccea8e57f0 P 4756eb5819a543f4bbf440e26deb8284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4756eb5819a543f4bbf440e26deb8284" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 34433 } } peers { permanent_uuid: "e70eeb6d81fd488b9c80b9c6de85eccc" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 41625 } } peers { permanent_uuid: "53a97834ac9e48e8b3c5ad90490c9d25" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 45699 } }
I20260806 22:48:00.241832 8936 ts_tablet_manager.cc:1428] T ee1b323fca4b4bdabf8a98ccea8e57f0 P e70eeb6d81fd488b9c80b9c6de85eccc: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:48:00.242090 8937 ts_tablet_manager.cc:1428] T ee1b323fca4b4bdabf8a98ccea8e57f0 P 4756eb5819a543f4bbf440e26deb8284: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:48:00.242117 8543 heartbeater.cc:503] Master 127.18.45.254:32823 was elected leader, sending a full tablet report...
I20260806 22:48:00.242532 8721 heartbeater.cc:503] Master 127.18.45.254:32823 was elected leader, sending a full tablet report...
W20260806 22:48:00.283092 8547 tablet.cc:2369] T ee1b323fca4b4bdabf8a98ccea8e57f0 P e70eeb6d81fd488b9c80b9c6de85eccc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:48:00.433840 8909 tablet.cc:2369] T ee1b323fca4b4bdabf8a98ccea8e57f0 P 53a97834ac9e48e8b3c5ad90490c9d25: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:48:00.484223 8722 tablet.cc:2369] T ee1b323fca4b4bdabf8a98ccea8e57f0 P 4756eb5819a543f4bbf440e26deb8284: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:48:00.515327 8946 raft_consensus.cc:482] T ee1b323fca4b4bdabf8a98ccea8e57f0 P 53a97834ac9e48e8b3c5ad90490c9d25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:48:00.515516 8946 raft_consensus.cc:504] T ee1b323fca4b4bdabf8a98ccea8e57f0 P 53a97834ac9e48e8b3c5ad90490c9d25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4756eb5819a543f4bbf440e26deb8284" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 34433 } } peers { permanent_uuid: "e70eeb6d81fd488b9c80b9c6de85eccc" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 41625 } } peers { permanent_uuid: "53a97834ac9e48e8b3c5ad90490c9d25" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 45699 } }
I20260806 22:48:00.516100 8946 leader_election.cc:288] T ee1b323fca4b4bdabf8a98ccea8e57f0 P 53a97834ac9e48e8b3c5ad90490c9d25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4756eb5819a543f4bbf440e26deb8284 (127.18.45.194:34433), e70eeb6d81fd488b9c80b9c6de85eccc (127.18.45.193:41625)
I20260806 22:48:00.519516 8910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:00.516393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.194:34433 (local address 127.18.45.195:58977)
0806 22:48:00.516457 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:00.516473 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 22:48:00.516561 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:48:00.516787 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:48:00.516789 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:48:00.516792 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:48:00.516895 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:48:00.516897 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:48:00.517637 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:48:00.517638 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:48:00.518467 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:48:00.518469 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:48:00.519199 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:48:00.519200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:48:00.519299 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:00.519303 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:48:00.519370 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 22:48:00.519421 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260806 22:48:00.519516 8934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:00.516466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.195:58977 (local address 127.18.45.194:34433)
0806 22:48:00.516556 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0806 22:48:00.516557 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:48:00.516569 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:48:00.516676 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:48:00.516677 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:48:00.516695 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:48:00.516776 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:48:00.516983 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:00.517537 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:00.518580 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:00.519112 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:00.519200 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:00.519215 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0806 22:48:00.519390 (+ 175us) server_negotiation.cc:299] Negotiation successful
0806 22:48:00.519439 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 22:48:00.520015 8949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:00.516402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.193:41625 (local address 127.18.45.195:52705)
0806 22:48:00.516876 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:00.516886 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:48:00.517018 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:48:00.517198 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:48:00.517200 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:48:00.517201 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:48:00.517305 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:48:00.517307 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:48:00.518061 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:48:00.518062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:48:00.518943 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:48:00.518945 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:48:00.519711 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:48:00.519712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:48:00.519810 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:00.519814 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:48:00.519886 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 22:48:00.519926 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"mutex_wait_us":227,"thread_start_us":176,"threads_started":1}
I20260806 22:48:00.520032 8933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:00.516575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.195:52705 (local address 127.18.45.193:41625)
0806 22:48:00.516660 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0806 22:48:00.516662 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:48:00.517010 (+ 348us) server_negotiation.cc:407] Connection header received
0806 22:48:00.517099 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:48:00.517100 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:48:00.517124 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:48:00.517193 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:48:00.517390 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:00.517946 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:00.519090 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:00.519539 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:00.519636 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:00.519648 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0806 22:48:00.519903 (+ 255us) server_negotiation.cc:299] Negotiation successful
0806 22:48:00.519958 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 22:48:00.520008 8661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee1b323fca4b4bdabf8a98ccea8e57f0" candidate_uuid: "53a97834ac9e48e8b3c5ad90490c9d25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4756eb5819a543f4bbf440e26deb8284" is_pre_election: true
I20260806 22:48:00.520273 8661 raft_consensus.cc:2454] T ee1b323fca4b4bdabf8a98ccea8e57f0 P 4756eb5819a543f4bbf440e26deb8284 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53a97834ac9e48e8b3c5ad90490c9d25 in term 0.
I20260806 22:48:00.520506 8481 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee1b323fca4b4bdabf8a98ccea8e57f0" candidate_uuid: "53a97834ac9e48e8b3c5ad90490c9d25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e70eeb6d81fd488b9c80b9c6de85eccc" is_pre_election: true
I20260806 22:48:00.520676 8481 raft_consensus.cc:2454] T ee1b323fca4b4bdabf8a98ccea8e57f0 P e70eeb6d81fd488b9c80b9c6de85eccc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53a97834ac9e48e8b3c5ad90490c9d25 in term 0.
I20260806 22:48:00.520643 8754 leader_election.cc:302] T ee1b323fca4b4bdabf8a98ccea8e57f0 P 53a97834ac9e48e8b3c5ad90490c9d25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4756eb5819a543f4bbf440e26deb8284, 53a97834ac9e48e8b3c5ad90490c9d25; no voters:
I20260806 22:48:00.520817 8946 raft_consensus.cc:2781] T ee1b323fca4b4bdabf8a98ccea8e57f0 P 53a97834ac9e48e8b3c5ad90490c9d25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:48:00.520898 8946 raft_consensus.cc:482] T ee1b323fca4b4bdabf8a98ccea8e57f0 P 53a97834ac9e48e8b3c5ad90490c9d25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:48:00.520931 8946 raft_consensus.cc:3037] T ee1b323fca4b4bdabf8a98ccea8e57f0 P 53a97834ac9e48e8b3c5ad90490c9d25 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:48:00.521896 8946 raft_consensus.cc:504] T ee1b323fca4b4bdabf8a98ccea8e57f0 P 53a97834ac9e48e8b3c5ad90490c9d25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4756eb5819a543f4bbf440e26deb8284" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 34433 } } peers { permanent_uuid: "e70eeb6d81fd488b9c80b9c6de85eccc" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 41625 } } peers { permanent_uuid: "53a97834ac9e48e8b3c5ad90490c9d25" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 45699 } }
I20260806 22:48:00.522082 8946 leader_election.cc:288] T ee1b323fca4b4bdabf8a98ccea8e57f0 P 53a97834ac9e48e8b3c5ad90490c9d25 [CANDIDATE]: Term 1 election: Requested vote from peers 4756eb5819a543f4bbf440e26deb8284 (127.18.45.194:34433), e70eeb6d81fd488b9c80b9c6de85eccc (127.18.45.193:41625)
I20260806 22:48:00.522267 8661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee1b323fca4b4bdabf8a98ccea8e57f0" candidate_uuid: "53a97834ac9e48e8b3c5ad90490c9d25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4756eb5819a543f4bbf440e26deb8284"
I20260806 22:48:00.522293 8481 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee1b323fca4b4bdabf8a98ccea8e57f0" candidate_uuid: "53a97834ac9e48e8b3c5ad90490c9d25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e70eeb6d81fd488b9c80b9c6de85eccc"
I20260806 22:48:00.522385 8661 raft_consensus.cc:3037] T ee1b323fca4b4bdabf8a98ccea8e57f0 P 4756eb5819a543f4bbf440e26deb8284 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:48:00.522384 8481 raft_consensus.cc:3037] T ee1b323fca4b4bdabf8a98ccea8e57f0 P e70eeb6d81fd488b9c80b9c6de85eccc [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:48:00.523623 8661 raft_consensus.cc:2454] T ee1b323fca4b4bdabf8a98ccea8e57f0 P 4756eb5819a543f4bbf440e26deb8284 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53a97834ac9e48e8b3c5ad90490c9d25 in term 1.
I20260806 22:48:00.523623 8481 raft_consensus.cc:2454] T ee1b323fca4b4bdabf8a98ccea8e57f0 P e70eeb6d81fd488b9c80b9c6de85eccc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53a97834ac9e48e8b3c5ad90490c9d25 in term 1.
I20260806 22:48:00.523855 8751 leader_election.cc:302] T ee1b323fca4b4bdabf8a98ccea8e57f0 P 53a97834ac9e48e8b3c5ad90490c9d25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53a97834ac9e48e8b3c5ad90490c9d25, e70eeb6d81fd488b9c80b9c6de85eccc; no voters:
I20260806 22:48:00.524004 8946 raft_consensus.cc:2781] T ee1b323fca4b4bdabf8a98ccea8e57f0 P 53a97834ac9e48e8b3c5ad90490c9d25 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:48:00.524233 8946 raft_consensus.cc:686] T ee1b323fca4b4bdabf8a98ccea8e57f0 P 53a97834ac9e48e8b3c5ad90490c9d25 [term 1 LEADER]: Becoming Leader. State: Replica: 53a97834ac9e48e8b3c5ad90490c9d25, State: Running, Role: LEADER
I20260806 22:48:00.524331 8946 consensus_queue.cc:237] T ee1b323fca4b4bdabf8a98ccea8e57f0 P 53a97834ac9e48e8b3c5ad90490c9d25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4756eb5819a543f4bbf440e26deb8284" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 34433 } } peers { permanent_uuid: "e70eeb6d81fd488b9c80b9c6de85eccc" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 41625 } } peers { permanent_uuid: "53a97834ac9e48e8b3c5ad90490c9d25" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 45699 } }
I20260806 22:48:00.525561 8290 catalog_manager.cc:5495] T ee1b323fca4b4bdabf8a98ccea8e57f0 P 53a97834ac9e48e8b3c5ad90490c9d25 reported cstate change: term changed from 0 to 1, leader changed from <none> to 53a97834ac9e48e8b3c5ad90490c9d25 (127.18.45.195). New cstate: current_term: 1 leader_uuid: "53a97834ac9e48e8b3c5ad90490c9d25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4756eb5819a543f4bbf440e26deb8284" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 34433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e70eeb6d81fd488b9c80b9c6de85eccc" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 41625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53a97834ac9e48e8b3c5ad90490c9d25" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 45699 } health_report { overall_health: HEALTHY } } }
I20260806 22:48:00.587651 8932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:00.584764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47018 (local address 127.18.45.195:45699)
0806 22:48:00.584842 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0806 22:48:00.584844 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:48:00.584857 (+ 13us) server_negotiation.cc:407] Connection header received
0806 22:48:00.584936 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:48:00.584937 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:48:00.584966 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:48:00.585058 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:48:00.585244 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:00.585804 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:00.586626 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:00.586853 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:00.586940 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:00.587174 (+ 234us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:48:00.587319 (+ 145us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:48:00.587433 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 22:48:00.587459 (+ 26us) server_negotiation.cc:299] Negotiation successful
0806 22:48:00.587488 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 22:48:00.590694 8661 raft_consensus.cc:1261] T ee1b323fca4b4bdabf8a98ccea8e57f0 P 4756eb5819a543f4bbf440e26deb8284 [term 1 FOLLOWER]: Refusing update from remote peer 53a97834ac9e48e8b3c5ad90490c9d25: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:48:00.590694 8481 raft_consensus.cc:1261] T ee1b323fca4b4bdabf8a98ccea8e57f0 P e70eeb6d81fd488b9c80b9c6de85eccc [term 1 FOLLOWER]: Refusing update from remote peer 53a97834ac9e48e8b3c5ad90490c9d25: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:48:00.591214 8950 consensus_queue.cc:1035] T ee1b323fca4b4bdabf8a98ccea8e57f0 P 53a97834ac9e48e8b3c5ad90490c9d25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e70eeb6d81fd488b9c80b9c6de85eccc" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 41625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:48:00.591503 8946 consensus_queue.cc:1035] T ee1b323fca4b4bdabf8a98ccea8e57f0 P 53a97834ac9e48e8b3c5ad90490c9d25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4756eb5819a543f4bbf440e26deb8284" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 34433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:48:00.594722 8956 mvcc.cc:205] Tried to move back new op lower bound from 7315687344495030272 to 7315687344228622336. Current Snapshot: MvccSnapshot[applied={T|T < 7315687344495030272}]
I20260806 22:48:00.595288 8961 mvcc.cc:205] Tried to move back new op lower bound from 7315687344495030272 to 7315687344228622336. Current Snapshot: MvccSnapshot[applied={T|T < 7315687344495030272}]
I20260806 22:48:00.596082 8959 mvcc.cc:205] Tried to move back new op lower bound from 7315687344495030272 to 7315687344228622336. Current Snapshot: MvccSnapshot[applied={T|T < 7315687344495030272}]
I20260806 22:48:00.598248 8290 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40904:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:48:00.598392 8290 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40904:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:48:00.599458 8290 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419: Sending DeleteTablet for 3 replicas of tablet ee1b323fca4b4bdabf8a98ccea8e57f0
I20260806 22:48:00.599874 8637 tablet_service.cc:1514] Processing DeleteTablet for tablet ee1b323fca4b4bdabf8a98ccea8e57f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:48:00 UTC) from {username='kudu', principal='kudu/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:39686
I20260806 22:48:00.599905 8455 tablet_service.cc:1514] Processing DeleteTablet for tablet ee1b323fca4b4bdabf8a98ccea8e57f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:48:00 UTC) from {username='kudu', principal='kudu/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:44336
I20260806 22:48:00.600224 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 8369
I20260806 22:48:00.612843 8820 tablet_service.cc:1514] Processing DeleteTablet for tablet ee1b323fca4b4bdabf8a98ccea8e57f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:48:00 UTC) from {username='kudu', principal='kudu/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:47016
I20260806 22:48:00.613835 8971 tablet_replica.cc:330] T ee1b323fca4b4bdabf8a98ccea8e57f0 P 53a97834ac9e48e8b3c5ad90490c9d25: stopping tablet replica
I20260806 22:48:00.614048 8971 raft_consensus.cc:2229] T ee1b323fca4b4bdabf8a98ccea8e57f0 P 53a97834ac9e48e8b3c5ad90490c9d25 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:48:00.614235 8971 raft_consensus.cc:2258] T ee1b323fca4b4bdabf8a98ccea8e57f0 P 53a97834ac9e48e8b3c5ad90490c9d25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:48:00.614900 8971 ts_tablet_manager.cc:1905] T ee1b323fca4b4bdabf8a98ccea8e57f0 P 53a97834ac9e48e8b3c5ad90490c9d25: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:48:00.616967 8971 ts_tablet_manager.cc:1918] T ee1b323fca4b4bdabf8a98ccea8e57f0 P 53a97834ac9e48e8b3c5ad90490c9d25: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:48:00.617091 8971 log.cc:1196] T ee1b323fca4b4bdabf8a98ccea8e57f0 P 53a97834ac9e48e8b3c5ad90490c9d25: Deleting WAL directory at /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-2/wal/wals/ee1b323fca4b4bdabf8a98ccea8e57f0
I20260806 22:48:00.617437 8971 ts_tablet_manager.cc:1939] T ee1b323fca4b4bdabf8a98ccea8e57f0 P 53a97834ac9e48e8b3c5ad90490c9d25: Deleting consensus metadata
I20260806 22:48:00.618091 8273 catalog_manager.cc:4841] TS 53a97834ac9e48e8b3c5ad90490c9d25 (127.18.45.195:45699): tablet ee1b323fca4b4bdabf8a98ccea8e57f0 (table test-table [id=9d187abacfb1498b9bd1864b7f5fb0d6]) successfully deleted
I20260806 22:48:00.619212 8969 tablet_replica.cc:330] T ee1b323fca4b4bdabf8a98ccea8e57f0 P 4756eb5819a543f4bbf440e26deb8284: stopping tablet replica
I20260806 22:48:00.619419 8969 raft_consensus.cc:2229] T ee1b323fca4b4bdabf8a98ccea8e57f0 P 4756eb5819a543f4bbf440e26deb8284 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:48:00.619624 8969 raft_consensus.cc:2258] T ee1b323fca4b4bdabf8a98ccea8e57f0 P 4756eb5819a543f4bbf440e26deb8284 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:48:00.620224 8969 ts_tablet_manager.cc:1905] T ee1b323fca4b4bdabf8a98ccea8e57f0 P 4756eb5819a543f4bbf440e26deb8284: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:48:00.621976 8969 ts_tablet_manager.cc:1918] T ee1b323fca4b4bdabf8a98ccea8e57f0 P 4756eb5819a543f4bbf440e26deb8284: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260806 22:48:00.622018 8271 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.45.193:41625 (error 108)
I20260806 22:48:00.622112 8969 log.cc:1196] T ee1b323fca4b4bdabf8a98ccea8e57f0 P 4756eb5819a543f4bbf440e26deb8284: Deleting WAL directory at /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-1/wal/wals/ee1b323fca4b4bdabf8a98ccea8e57f0
I20260806 22:48:00.622431 8969 ts_tablet_manager.cc:1939] T ee1b323fca4b4bdabf8a98ccea8e57f0 P 4756eb5819a543f4bbf440e26deb8284: Deleting consensus metadata
I20260806 22:48:00.622452 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 8551
I20260806 22:48:00.622730 8928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:00.622577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.193:41625 (local address 127.0.0.1:44340)
0806 22:48:00.622629 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:00.622680 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.193:41625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260806 22:48:00.622848 8271 catalog_manager.cc:4584] TS e70eeb6d81fd488b9c80b9c6de85eccc (127.18.45.193:41625): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ee1b323fca4b4bdabf8a98ccea8e57f0: Network error: Client connection negotiation failed: client connection to 127.18.45.193:41625: connect: Connection refused (error 111)
I20260806 22:48:00.633651 8928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:00.633444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.194:34433 (local address 127.0.0.1:39692)
0806 22:48:00.633529 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:00.633572 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.194:34433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260806 22:48:00.633746 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 8728
I20260806 22:48:00.647740 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 8253
I20260806 22:48:20.663153 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:32823
--webserver_interface=127.18.45.254
--webserver_port=41899
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:41985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.45.254:32823
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:48:20.761015 8974 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:48:20.761291 8974 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:48:20.761317 8974 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:48:20.762552 8974 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 22:48:20.762598 8974 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 22:48:20.762611 8974 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 22:48:20.762624 8974 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:48:20.762684 8974 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:48:20.762702 8974 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:48:20.762727 8974 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:48:20.762750 8974 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:48:20.764489 8974 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:41985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.18.45.254:32823
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:32823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.45.254
--webserver_port=41899
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8974
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:48:20.764819 8974 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:48:20.765113 8974 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:48:20.776743 8983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:48:20.776758 8984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:48:20.776769 8974 server_base.cc:1034] running on GCE node
W20260806 22:48:20.776746 8986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:48:20.777401 8974 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:48:20.777767 8974 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:48:20.778935 8974 hybrid_clock.cc:648] HybridClock initialized: now 1786056500778913 us; error 43 us; skew 500 ppm
Aug 06 22:48:20 dist-test-slave-h3ft krb5kdc[8237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056500, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:48:20.782074 8974 init.cc:377] Logged in from keytab as kudu/127.18.45.254@KRBTEST.COM (short username kudu)
I20260806 22:48:20.782644 8974 webserver.cc:460] Webserver started at http://127.18.45.254:41899/ using document root <none> and password file <none>
I20260806 22:48:20.782938 8974 fs_manager.cc:359] Metadata directory not provided
I20260806 22:48:20.783054 8974 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:48:20.784776 8974 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:48:20.785692 8994 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:48:20.785926 8974 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:48:20.786095 8974 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/master-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/master-0/wal
uuid: "2817a20f1830402dbc70359306e75419"
format_stamp: "Formatted at 2026-08-06 22:47:59 on dist-test-slave-h3ft"
server_key: "2e9805f58a1ffd833fa20b15e90cddc4"
server_key_iv: "95356632211c9ff1b9aac382ea71a14c"
server_key_version: "encryptionkey@0"
I20260806 22:48:20.786597 8974 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:48:20.802610 8974 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:48:20.805104 8974 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:48:20.805292 8974 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:48:20.810698 8974 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.254:32823
I20260806 22:48:20.810740 9067 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.254:32823 every 8 connection(s)
I20260806 22:48:20.811309 8974 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
I20260806 22:48:20.812248 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 8974
I20260806 22:48:20.812677 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.193:41625
--local_ip_for_outbound_sockets=127.18.45.193
--tserver_master_addrs=127.18.45.254:32823
--webserver_port=38949
--webserver_interface=127.18.45.193
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab
--principal=kudu/127.18.45.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:41985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:48:20.814028 9068 sys_catalog.cc:263] Verifying existing consensus state
I20260806 22:48:20.815366 9068 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419: Bootstrap starting.
I20260806 22:48:20.827965 9068 log.cc:824] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419: Log is configured to *not* fsync() on all Append() calls
I20260806 22:48:20.830742 9068 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:48:20.831219 9068 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419: Bootstrap complete.
I20260806 22:48:20.834263 9068 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2817a20f1830402dbc70359306e75419" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 32823 } }
I20260806 22:48:20.834553 9068 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2817a20f1830402dbc70359306e75419, State: Initialized, Role: FOLLOWER
I20260806 22:48:20.834780 9068 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2817a20f1830402dbc70359306e75419" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 32823 } }
I20260806 22:48:20.834864 9068 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:48:20.834941 9068 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:48:20.835023 9068 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419 [term 1 FOLLOWER]: Advancing to term 2
I20260806 22:48:20.836365 9068 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2817a20f1830402dbc70359306e75419" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 32823 } }
I20260806 22:48:20.836474 9068 leader_election.cc:302] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2817a20f1830402dbc70359306e75419; no voters:
I20260806 22:48:20.836747 9068 leader_election.cc:288] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 22:48:20.836797 9072 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 22:48:20.837015 9072 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419 [term 2 LEADER]: Becoming Leader. State: Replica: 2817a20f1830402dbc70359306e75419, State: Running, Role: LEADER
I20260806 22:48:20.837169 9068 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:48:20.837196 9072 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2817a20f1830402dbc70359306e75419" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 32823 } }
I20260806 22:48:20.837606 9072 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2817a20f1830402dbc70359306e75419. Latest consensus state: current_term: 2 leader_uuid: "2817a20f1830402dbc70359306e75419" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2817a20f1830402dbc70359306e75419" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 32823 } } }
I20260806 22:48:20.837610 9075 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "2817a20f1830402dbc70359306e75419" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2817a20f1830402dbc70359306e75419" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 32823 } } }
I20260806 22:48:20.837882 9072 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419 [sys.catalog]: This master's current role is: LEADER
I20260806 22:48:20.837898 9075 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419 [sys.catalog]: This master's current role is: LEADER
I20260806 22:48:20.838311 9084 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:48:20.839588 9084 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:48:20.839794 9084 catalog_manager.cc:1259] Loaded cluster ID: 1b4274433ca54a8183f72fe367b61dba
I20260806 22:48:20.840071 9084 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:48:20.840991 9084 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:48:20.841133 9084 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419: Loaded TSK: 0
I20260806 22:48:20.857756 9084 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419: Generated new TSK 1
I20260806 22:48:20.857966 9084 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 22:48:20.933610 9070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:48:20.933941 9070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:48:20.933990 9070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:48:20.935328 9070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:48:20.935405 9070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:48:20.935453 9070 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.193
I20260806 22:48:20.937144 9070 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:41985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab
--principal=kudu/127.18.45.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.193:41625
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.45.193
--webserver_port=38949
--tserver_master_addrs=127.18.45.254:32823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9070
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.193
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:48:20.937539 9070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:48:20.937860 9070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:48:20.944223 9106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:48:20.944245 9103 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:48:20.944223 9104 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:48:20.944658 9070 server_base.cc:1034] running on GCE node
I20260806 22:48:20.944921 9070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:48:20.945195 9070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:48:20.946408 9070 hybrid_clock.cc:648] HybridClock initialized: now 1786056500946388 us; error 36 us; skew 500 ppm
Aug 06 22:48:20 dist-test-slave-h3ft krb5kdc[8237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056500, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:48:20.949951 9070 init.cc:377] Logged in from keytab as kudu/127.18.45.193@KRBTEST.COM (short username kudu)
I20260806 22:48:20.950624 9070 webserver.cc:460] Webserver started at http://127.18.45.193:38949/ using document root <none> and password file <none>
I20260806 22:48:20.950953 9070 fs_manager.cc:359] Metadata directory not provided
I20260806 22:48:20.951087 9070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:48:20.952883 9070 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:48:20.953770 9116 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:48:20.954059 9070 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:48:20.954208 9070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-0/wal
uuid: "e70eeb6d81fd488b9c80b9c6de85eccc"
format_stamp: "Formatted at 2026-08-06 22:47:59 on dist-test-slave-h3ft"
server_key: "cd42bc14687e788551e73e3f92592154"
server_key_iv: "2e1e4ec8c02c735da89f96ce03bed992"
server_key_version: "encryptionkey@0"
I20260806 22:48:20.954667 9070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:48:20.970077 9070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:48:20.973047 9070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:48:20.973235 9070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:48:20.973558 9070 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:48:20.974148 9126 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 22:48:20.976051 9070 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 22:48:20.976114 9070 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260806 22:48:20.976146 9070 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 22:48:20.976959 9070 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 22:48:20.977005 9070 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260806 22:48:20.977063 9126 tablet_bootstrap.cc:492] T ee1b323fca4b4bdabf8a98ccea8e57f0 P e70eeb6d81fd488b9c80b9c6de85eccc: Bootstrap starting.
I20260806 22:48:20.987462 9070 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.193:41625
I20260806 22:48:20.987707 9267 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.193:41625 every 8 connection(s)
I20260806 22:48:20.988166 9070 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
I20260806 22:48:20.991449 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 9070
I20260806 22:48:20.991827 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.194:34433
--local_ip_for_outbound_sockets=127.18.45.194
--tserver_master_addrs=127.18.45.254:32823
--webserver_port=38907
--webserver_interface=127.18.45.194
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab
--principal=kudu/127.18.45.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:41985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:48:20.993857 9126 log.cc:824] T ee1b323fca4b4bdabf8a98ccea8e57f0 P e70eeb6d81fd488b9c80b9c6de85eccc: Log is configured to *not* fsync() on all Append() calls
I20260806 22:48:20.997570 9126 tablet_bootstrap.cc:492] T ee1b323fca4b4bdabf8a98ccea8e57f0 P e70eeb6d81fd488b9c80b9c6de85eccc: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:48:20.998018 9126 tablet_bootstrap.cc:492] T ee1b323fca4b4bdabf8a98ccea8e57f0 P e70eeb6d81fd488b9c80b9c6de85eccc: Bootstrap complete.
Aug 06 22:48:20 dist-test-slave-h3ft krb5kdc[8237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056500, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.193@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:48:20.999707 9126 ts_tablet_manager.cc:1397] T ee1b323fca4b4bdabf8a98ccea8e57f0 P e70eeb6d81fd488b9c80b9c6de85eccc: Time spent bootstrapping tablet: real 0.023s user 0.005s sys 0.017s
I20260806 22:48:21.001698 9126 raft_consensus.cc:348] T ee1b323fca4b4bdabf8a98ccea8e57f0 P e70eeb6d81fd488b9c80b9c6de85eccc [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4756eb5819a543f4bbf440e26deb8284" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 34433 } } peers { permanent_uuid: "e70eeb6d81fd488b9c80b9c6de85eccc" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 41625 } } peers { permanent_uuid: "53a97834ac9e48e8b3c5ad90490c9d25" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 45699 } }
I20260806 22:48:21.002216 9126 raft_consensus.cc:729] T ee1b323fca4b4bdabf8a98ccea8e57f0 P e70eeb6d81fd488b9c80b9c6de85eccc [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e70eeb6d81fd488b9c80b9c6de85eccc, State: Initialized, Role: FOLLOWER
I20260806 22:48:21.002460 9126 consensus_queue.cc:260] T ee1b323fca4b4bdabf8a98ccea8e57f0 P e70eeb6d81fd488b9c80b9c6de85eccc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4756eb5819a543f4bbf440e26deb8284" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 34433 } } peers { permanent_uuid: "e70eeb6d81fd488b9c80b9c6de85eccc" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 41625 } } peers { permanent_uuid: "53a97834ac9e48e8b3c5ad90490c9d25" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 45699 } }
I20260806 22:48:21.002887 9126 ts_tablet_manager.cc:1428] T ee1b323fca4b4bdabf8a98ccea8e57f0 P e70eeb6d81fd488b9c80b9c6de85eccc: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260806 22:48:21.006006 9272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:20.989248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:39499 (local address 127.18.45.254:32823)
0806 22:48:20.990536 (+ 1288us) server_negotiation.cc:206] Beginning negotiation
0806 22:48:20.990547 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:48:20.990595 (+ 48us) server_negotiation.cc:407] Connection header received
0806 22:48:20.990753 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:48:20.990760 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:48:20.990824 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:48:20.991033 (+ 209us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:48:20.992534 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:20.994215 (+ 1681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:20.995888 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:20.996514 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:20.996630 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:21.000391 (+ 3761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:48:21.000396 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:48:21.000397 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:48:21.000412 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:48:21.003520 (+ 3108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:48:21.004170 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:48:21.004174 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:48:21.004174 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:48:21.004310 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:48:21.004674 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:48:21.004675 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:48:21.004676 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:48:21.005010 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:48:21.005089 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 22:48:21.005524 (+ 435us) server_negotiation.cc:299] Negotiation successful
0806 22:48:21.005679 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1227,"thread_start_us":116,"threads_started":1}
I20260806 22:48:21.006027 9271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:20.989115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:32823 (local address 127.18.45.193:39499)
0806 22:48:20.989462 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:20.989522 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 22:48:20.990273 (+ 751us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:48:20.991352 (+ 1079us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:48:20.991358 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:48:20.991934 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:48:20.992321 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:48:20.992337 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:48:20.994390 (+ 2053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:48:20.994391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:48:20.995769 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:48:20.995772 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:48:20.996771 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:48:20.996773 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:48:20.996956 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:20.997917 (+ 961us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:48:20.997928 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:48:21.000215 (+ 2287us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:48:21.003635 (+ 3420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:48:21.003643 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:48:21.003646 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:48:21.004046 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:48:21.004400 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:48:21.004401 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:48:21.004401 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:48:21.004581 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:48:21.005092 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:48:21.005101 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:48:21.005299 (+ 198us) client_negotiation.cc:769] Sending connection context
0806 22:48:21.005441 (+ 142us) client_negotiation.cc:240] Negotiation successful
0806 22:48:21.005679 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":112,"threads_started":1}
I20260806 22:48:21.007274 9268 heartbeater.cc:347] Connected to a master server at 127.18.45.254:32823
I20260806 22:48:21.007391 9268 heartbeater.cc:464] Registering TS with master...
I20260806 22:48:21.007608 9268 heartbeater.cc:511] Master 127.18.45.254:32823 requested a full tablet report, sending...
I20260806 22:48:21.008603 9014 ts_manager.cc:194] Registered new tserver with Master: e70eeb6d81fd488b9c80b9c6de85eccc (127.18.45.193:41625)
I20260806 22:48:21.010118 9014 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.45.193@KRBTEST.COM'} at 127.18.45.193:39499
I20260806 22:48:21.011267 9268 heartbeater.cc:503] Master 127.18.45.254:32823 was elected leader, sending a full tablet report...
I20260806 22:48:21.015798 9278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:21.009798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42032 (local address 127.18.45.193:41625)
0806 22:48:21.011403 (+ 1605us) server_negotiation.cc:206] Beginning negotiation
0806 22:48:21.011408 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:48:21.011427 (+ 19us) server_negotiation.cc:407] Connection header received
0806 22:48:21.011464 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:48:21.011466 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:48:21.011484 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:48:21.011774 (+ 290us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:48:21.011821 (+ 47us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:21.012962 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:21.014516 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:21.015294 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:21.015373 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:21.015445 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 22:48:21.015666 (+ 221us) server_negotiation.cc:299] Negotiation successful
0806 22:48:21.015712 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1570,"thread_start_us":105,"threads_started":1}
I20260806 22:48:21.015826 9277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:21.009469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.193:41625 (local address 127.0.0.1:42032)
0806 22:48:21.011139 (+ 1670us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:21.011182 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 22:48:21.011263 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:48:21.011590 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:48:21.011593 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:48:21.011596 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:48:21.011705 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:48:21.011708 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:48:21.013065 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:48:21.013066 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:48:21.014405 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:48:21.014411 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:48:21.015396 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:48:21.015398 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:48:21.015525 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:21.015546 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 22:48:21.015643 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 22:48:21.015716 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1624,"thread_start_us":146,"threads_started":1}
I20260806 22:48:21.016733 9176 tablet_service.cc:1514] Processing DeleteTablet for tablet ee1b323fca4b4bdabf8a98ccea8e57f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:48:00 UTC) from {username='kudu', principal='kudu/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:42032
I20260806 22:48:21.017458 9279 tablet_replica.cc:330] stopping tablet replica
I20260806 22:48:21.017592 9279 raft_consensus.cc:2229] T ee1b323fca4b4bdabf8a98ccea8e57f0 P e70eeb6d81fd488b9c80b9c6de85eccc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:48:21.017673 9279 raft_consensus.cc:2258] T ee1b323fca4b4bdabf8a98ccea8e57f0 P e70eeb6d81fd488b9c80b9c6de85eccc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:48:21.018061 9279 ts_tablet_manager.cc:1905] T ee1b323fca4b4bdabf8a98ccea8e57f0 P e70eeb6d81fd488b9c80b9c6de85eccc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:48:21.020805 9279 ts_tablet_manager.cc:1918] T ee1b323fca4b4bdabf8a98ccea8e57f0 P e70eeb6d81fd488b9c80b9c6de85eccc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:48:21.020902 9279 log.cc:1196] T ee1b323fca4b4bdabf8a98ccea8e57f0 P e70eeb6d81fd488b9c80b9c6de85eccc: Deleting WAL directory at /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-0/wal/wals/ee1b323fca4b4bdabf8a98ccea8e57f0
I20260806 22:48:21.021168 9279 ts_tablet_manager.cc:1939] T ee1b323fca4b4bdabf8a98ccea8e57f0 P e70eeb6d81fd488b9c80b9c6de85eccc: Deleting consensus metadata
I20260806 22:48:21.021739 8995 catalog_manager.cc:4841] TS e70eeb6d81fd488b9c80b9c6de85eccc (127.18.45.193:41625): tablet ee1b323fca4b4bdabf8a98ccea8e57f0 (table test-table [id=9d187abacfb1498b9bd1864b7f5fb0d6]) successfully deleted
W20260806 22:48:21.098445 9273 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:48:21.098769 9273 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:48:21.098814 9273 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:48:21.100160 9273 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:48:21.100241 9273 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:48:21.100287 9273 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.194
I20260806 22:48:21.102267 9273 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:41985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab
--principal=kudu/127.18.45.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.194:34433
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.45.194
--webserver_port=38907
--tserver_master_addrs=127.18.45.254:32823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9273
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.194
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:48:21.102607 9273 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:48:21.102924 9273 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:48:21.109261 9291 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:48:21.109277 9284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:48:21.109300 9273 server_base.cc:1034] running on GCE node
W20260806 22:48:21.109272 9287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:48:21.109694 9273 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:48:21.110019 9273 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:48:21.111186 9273 hybrid_clock.cc:648] HybridClock initialized: now 1786056501111157 us; error 46 us; skew 500 ppm
Aug 06 22:48:21 dist-test-slave-h3ft krb5kdc[8237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056501, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:48:21.114745 9273 init.cc:377] Logged in from keytab as kudu/127.18.45.194@KRBTEST.COM (short username kudu)
I20260806 22:48:21.115463 9273 webserver.cc:460] Webserver started at http://127.18.45.194:38907/ using document root <none> and password file <none>
I20260806 22:48:21.115788 9273 fs_manager.cc:359] Metadata directory not provided
I20260806 22:48:21.115875 9273 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:48:21.117635 9273 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 22:48:21.118515 9301 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:48:21.118741 9273 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:48:21.118921 9273 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-1/wal
uuid: "4756eb5819a543f4bbf440e26deb8284"
format_stamp: "Formatted at 2026-08-06 22:47:59 on dist-test-slave-h3ft"
server_key: "0889662080420627d084d3b1538d3ef9"
server_key_iv: "8851ad8f354a9ce5e6d2bd0c3067b1b7"
server_key_version: "encryptionkey@0"
I20260806 22:48:21.119519 9273 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:48:21.134240 9273 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:48:21.137177 9273 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:48:21.137395 9273 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:48:21.137779 9273 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:48:21.138197 9273 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:48:21.138263 9273 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:48:21.138310 9273 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:48:21.138350 9273 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:48:21.146692 9273 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.194:34433
I20260806 22:48:21.146747 9461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.194:34433 every 8 connection(s)
I20260806 22:48:21.147264 9273 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
I20260806 22:48:21.152570 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 9273
I20260806 22:48:21.152936 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.195:45699
--local_ip_for_outbound_sockets=127.18.45.195
--tserver_master_addrs=127.18.45.254:32823
--webserver_port=42291
--webserver_interface=127.18.45.195
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab
--principal=kudu/127.18.45.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:41985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 22:48:21 dist-test-slave-h3ft krb5kdc[8237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056501, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.194@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:48:21.160012 9272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:21.148271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:36661 (local address 127.18.45.254:32823)
0806 22:48:21.148370 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0806 22:48:21.148372 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:48:21.148600 (+ 228us) server_negotiation.cc:407] Connection header received
0806 22:48:21.149216 (+ 616us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:48:21.149218 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:48:21.149237 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:48:21.149309 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:48:21.150531 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:21.151194 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:21.152461 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:21.152827 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:21.152922 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:21.155930 (+ 3008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:48:21.155932 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:48:21.155932 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:48:21.155949 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:48:21.157930 (+ 1981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:48:21.158595 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:48:21.158596 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:48:21.158597 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:48:21.158685 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:48:21.159098 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:48:21.159099 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:48:21.159099 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:48:21.159288 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:48:21.159377 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 22:48:21.159695 (+ 318us) server_negotiation.cc:299] Negotiation successful
0806 22:48:21.159827 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 22:48:21.160044 9464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:21.148204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:32823 (local address 127.18.45.194:36661)
0806 22:48:21.148412 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:21.148466 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 22:48:21.149110 (+ 644us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:48:21.149419 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:48:21.149424 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:48:21.150139 (+ 715us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:48:21.150408 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:48:21.150414 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:48:21.151315 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:48:21.151316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:48:21.152354 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:48:21.152358 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:48:21.152939 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:48:21.152941 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:48:21.153160 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:21.153923 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:48:21.153933 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:48:21.155811 (+ 1878us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:48:21.158133 (+ 2322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:48:21.158143 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:48:21.158147 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:48:21.158502 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:48:21.158775 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:48:21.158776 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:48:21.158776 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:48:21.158936 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:48:21.159372 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:48:21.159379 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:48:21.159533 (+ 154us) client_negotiation.cc:769] Sending connection context
0806 22:48:21.159645 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 22:48:21.159814 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":117,"threads_started":1}
I20260806 22:48:21.160722 9462 heartbeater.cc:347] Connected to a master server at 127.18.45.254:32823
I20260806 22:48:21.160871 9462 heartbeater.cc:464] Registering TS with master...
I20260806 22:48:21.161144 9462 heartbeater.cc:511] Master 127.18.45.254:32823 requested a full tablet report, sending...
I20260806 22:48:21.161738 9014 ts_manager.cc:194] Registered new tserver with Master: 4756eb5819a543f4bbf440e26deb8284 (127.18.45.194:34433)
I20260806 22:48:21.162356 9014 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.45.194@KRBTEST.COM'} at 127.18.45.194:36661
W20260806 22:48:21.252912 9465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:48:21.253183 9465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:48:21.253211 9465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:48:21.254421 9465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:48:21.254459 9465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:48:21.254485 9465 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.195
I20260806 22:48:21.256028 9465 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:41985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab
--principal=kudu/127.18.45.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.195:45699
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.45.195
--webserver_port=42291
--tserver_master_addrs=127.18.45.254:32823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9465
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.195
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:48:21.256273 9465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:48:21.256507 9465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:48:21.262478 9479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:48:21.262503 9473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:48:21.262485 9476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:48:21.262814 9465 server_base.cc:1034] running on GCE node
I20260806 22:48:21.263141 9465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:48:21.263465 9465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:48:21.264653 9465 hybrid_clock.cc:648] HybridClock initialized: now 1786056501264638 us; error 34 us; skew 500 ppm
Aug 06 22:48:21 dist-test-slave-h3ft krb5kdc[8237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056501, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:48:21.267863 9465 init.cc:377] Logged in from keytab as kudu/127.18.45.195@KRBTEST.COM (short username kudu)
I20260806 22:48:21.268435 9465 webserver.cc:460] Webserver started at http://127.18.45.195:42291/ using document root <none> and password file <none>
I20260806 22:48:21.268699 9465 fs_manager.cc:359] Metadata directory not provided
I20260806 22:48:21.268803 9465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:48:21.270488 9465 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:48:21.271341 9494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:48:21.271555 9465 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:48:21.271678 9465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-2/wal
uuid: "53a97834ac9e48e8b3c5ad90490c9d25"
format_stamp: "Formatted at 2026-08-06 22:48:00 on dist-test-slave-h3ft"
server_key: "8b3fc2654f66064554342d0522709aaf"
server_key_iv: "9681cd6fa5631a2d288a7784fd80f65b"
server_key_version: "encryptionkey@0"
I20260806 22:48:21.272148 9465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:48:21.282025 9465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:48:21.284791 9465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:48:21.284983 9465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:48:21.285259 9465 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:48:21.285671 9465 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:48:21.285744 9465 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:48:21.285789 9465 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:48:21.285816 9465 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:48:21.293501 9465 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.195:45699
I20260806 22:48:21.293591 9650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.195:45699 every 8 connection(s)
I20260806 22:48:21.294021 9465 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
Aug 06 22:48:21 dist-test-slave-h3ft krb5kdc[8237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056501, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.195@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:48:21.301746 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 9465
I20260806 22:48:21.305055 9272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:21.295063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.195:39425 (local address 127.18.45.254:32823)
0806 22:48:21.295138 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 22:48:21.295141 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:48:21.295367 (+ 226us) server_negotiation.cc:407] Connection header received
0806 22:48:21.295984 (+ 617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:48:21.295986 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:48:21.296000 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:48:21.296068 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:48:21.297222 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:21.297779 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:21.298911 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:21.299201 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:21.299268 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:21.301740 (+ 2472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:48:21.301743 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:48:21.301743 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:48:21.301752 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:48:21.303505 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:48:21.303967 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:48:21.303969 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:48:21.303969 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:48:21.304037 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:48:21.304320 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:48:21.304321 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:48:21.304321 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:48:21.304456 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:48:21.304536 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 22:48:21.304844 (+ 308us) server_negotiation.cc:299] Negotiation successful
0806 22:48:21.304944 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 22:48:21.305297 9653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:21.294984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:32823 (local address 127.18.45.195:39425)
0806 22:48:21.295218 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:21.295262 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 22:48:21.295857 (+ 595us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:48:21.296158 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:48:21.296162 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:48:21.296833 (+ 671us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:48:21.297101 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:48:21.297107 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:48:21.297872 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:48:21.297873 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:48:21.298776 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:48:21.298779 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:48:21.299287 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:48:21.299288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:48:21.299421 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:21.300018 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:48:21.300027 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:48:21.301629 (+ 1602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:48:21.303612 (+ 1983us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:48:21.303621 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:48:21.303624 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:48:21.303883 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:48:21.304112 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:48:21.304113 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:48:21.304114 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:48:21.304241 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:48:21.304525 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:48:21.304533 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:48:21.304695 (+ 162us) client_negotiation.cc:769] Sending connection context
0806 22:48:21.304820 (+ 125us) client_negotiation.cc:240] Negotiation successful
0806 22:48:21.305019 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":110,"threads_started":1}
I20260806 22:48:21.305931 9651 heartbeater.cc:347] Connected to a master server at 127.18.45.254:32823
I20260806 22:48:21.306064 9651 heartbeater.cc:464] Registering TS with master...
I20260806 22:48:21.306296 9651 heartbeater.cc:511] Master 127.18.45.254:32823 requested a full tablet report, sending...
I20260806 22:48:21.306914 9014 ts_manager.cc:194] Registered new tserver with Master: 53a97834ac9e48e8b3c5ad90490c9d25 (127.18.45.195:45699)
I20260806 22:48:21.307368 9014 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.45.195@KRBTEST.COM'} at 127.18.45.195:39425
I20260806 22:48:21.309952 9654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:21.302275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35234 (local address 127.18.45.254:32823)
0806 22:48:21.302411 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 22:48:21.302413 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:48:21.302572 (+ 159us) server_negotiation.cc:407] Connection header received
0806 22:48:21.302660 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:48:21.302661 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:48:21.302674 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:48:21.302771 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:48:21.303498 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:21.304036 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:21.304868 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:21.305298 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:21.305390 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:21.307125 (+ 1735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:48:21.307126 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:48:21.307127 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:48:21.307135 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:48:21.308675 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:48:21.309080 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:48:21.309081 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:48:21.309081 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:48:21.309154 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:48:21.309366 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:48:21.309367 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:48:21.309368 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:48:21.309481 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:48:21.309640 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 22:48:21.309677 (+ 37us) server_negotiation.cc:299] Negotiation successful
0806 22:48:21.309757 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":84,"threads_started":1}
I20260806 22:48:21.310901 18615 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:48:21.313910 9654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:21.311354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35246 (local address 127.18.45.254:32823)
0806 22:48:21.311422 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 22:48:21.311424 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:48:21.311500 (+ 76us) server_negotiation.cc:407] Connection header received
0806 22:48:21.311577 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:48:21.311578 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:48:21.311614 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:48:21.311674 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:48:21.311775 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:21.312247 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:21.313152 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:21.313368 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:21.313434 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:21.313583 (+ 149us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:48:21.313636 (+ 53us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0806 22:48:21.313778 (+ 142us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35246: token expired
Metrics: {"server-negotiator.queue_time_us":42}
W20260806 22:48:21.313984 9654 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35246: token expired
W20260806 22:48:21.314162 8917 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.18.45.254:32823)
I20260806 22:48:21.321205 9654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:21.314419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35258 (local address 127.18.45.254:32823)
0806 22:48:21.314478 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 22:48:21.314480 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:48:21.314487 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:48:21.314506 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:48:21.314507 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:48:21.314515 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:48:21.314560 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:48:21.315141 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:21.315601 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:21.316431 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:21.316649 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:21.316713 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:21.318030 (+ 1317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:48:21.318032 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:48:21.318032 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:48:21.318040 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:48:21.319898 (+ 1858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:48:21.320357 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:48:21.320358 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:48:21.320359 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:48:21.320427 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:48:21.320639 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:48:21.320639 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:48:21.320640 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:48:21.320810 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:48:21.320922 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 22:48:21.321020 (+ 98us) server_negotiation.cc:299] Negotiation successful
0806 22:48:21.321101 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 22:48:21.322896 9014 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35258:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:48:21.325183 9014 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:48:21.328262 9176 tablet_service.cc:1467] Processing CreateTablet for tablet bf5ce45c1617423094e79ec99b51cdab (DEFAULT_TABLE table=test-table [id=b44df97c4a944671a02f6f5077b85093]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:48:21.328557 9176 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf5ce45c1617423094e79ec99b51cdab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:48:21.330371 9126 tablet_bootstrap.cc:492] T bf5ce45c1617423094e79ec99b51cdab P e70eeb6d81fd488b9c80b9c6de85eccc: Bootstrap starting.
I20260806 22:48:21.330793 9126 tablet_bootstrap.cc:654] T bf5ce45c1617423094e79ec99b51cdab P e70eeb6d81fd488b9c80b9c6de85eccc: Neither blocks nor log segments found. Creating new log.
I20260806 22:48:21.331573 9126 tablet_bootstrap.cc:492] T bf5ce45c1617423094e79ec99b51cdab P e70eeb6d81fd488b9c80b9c6de85eccc: No bootstrap required, opened a new log
I20260806 22:48:21.331632 9126 ts_tablet_manager.cc:1397] T bf5ce45c1617423094e79ec99b51cdab P e70eeb6d81fd488b9c80b9c6de85eccc: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 22:48:21.331772 9126 raft_consensus.cc:348] T bf5ce45c1617423094e79ec99b51cdab P e70eeb6d81fd488b9c80b9c6de85eccc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4756eb5819a543f4bbf440e26deb8284" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 34433 } } peers { permanent_uuid: "53a97834ac9e48e8b3c5ad90490c9d25" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 45699 } } peers { permanent_uuid: "e70eeb6d81fd488b9c80b9c6de85eccc" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 41625 } }
I20260806 22:48:21.331859 9126 raft_consensus.cc:374] T bf5ce45c1617423094e79ec99b51cdab P e70eeb6d81fd488b9c80b9c6de85eccc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:48:21.331916 9126 raft_consensus.cc:729] T bf5ce45c1617423094e79ec99b51cdab P e70eeb6d81fd488b9c80b9c6de85eccc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e70eeb6d81fd488b9c80b9c6de85eccc, State: Initialized, Role: FOLLOWER
I20260806 22:48:21.331970 9126 consensus_queue.cc:260] T bf5ce45c1617423094e79ec99b51cdab P e70eeb6d81fd488b9c80b9c6de85eccc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4756eb5819a543f4bbf440e26deb8284" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 34433 } } peers { permanent_uuid: "53a97834ac9e48e8b3c5ad90490c9d25" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 45699 } } peers { permanent_uuid: "e70eeb6d81fd488b9c80b9c6de85eccc" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 41625 } }
I20260806 22:48:21.332088 9126 ts_tablet_manager.cc:1428] T bf5ce45c1617423094e79ec99b51cdab P e70eeb6d81fd488b9c80b9c6de85eccc: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 22:48:21.332394 9277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:21.327706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.194:34433 (local address 127.0.0.1:50160)
0806 22:48:21.327789 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:21.327806 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 22:48:21.327849 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:48:21.328421 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:48:21.328422 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:48:21.328425 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:48:21.328499 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:48:21.328501 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:48:21.329858 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:48:21.329859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:48:21.331070 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:48:21.331074 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:48:21.332130 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:48:21.332131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:48:21.332206 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:21.332213 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 22:48:21.332247 (+ 34us) client_negotiation.cc:240] Negotiation successful
0806 22:48:21.332302 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59,"mutex_wait_us":3}
I20260806 22:48:21.332650 9658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:21.327784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.195:45699 (local address 127.0.0.1:38070)
0806 22:48:21.327961 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:21.327975 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:48:21.328038 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:48:21.328649 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:48:21.328651 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:48:21.328653 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:48:21.328708 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:48:21.328710 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:48:21.330036 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:48:21.330038 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:48:21.331069 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:48:21.331073 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:48:21.332362 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:48:21.332363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:48:21.332433 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:21.332437 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:48:21.332501 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 22:48:21.332553 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":30,"thread_start_us":130,"threads_started":1}
I20260806 22:48:21.332819 9662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:21.328010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50160 (local address 127.18.45.194:34433)
0806 22:48:21.328231 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0806 22:48:21.328237 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:48:21.328247 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:48:21.328291 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:48:21.328293 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:48:21.328308 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:48:21.328419 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:48:21.328549 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:21.329751 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:21.331159 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:21.332018 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:21.332530 (+ 512us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:21.332629 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 22:48:21.332663 (+ 34us) server_negotiation.cc:299] Negotiation successful
0806 22:48:21.332718 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":128,"threads_started":1}
I20260806 22:48:21.333052 9660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:21.327894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38070 (local address 127.18.45.195:45699)
0806 22:48:21.328320 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0806 22:48:21.328325 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:48:21.328338 (+ 13us) server_negotiation.cc:407] Connection header received
0806 22:48:21.328388 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:48:21.328390 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:48:21.328408 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:48:21.328518 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:48:21.328807 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:21.329952 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:21.331417 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:21.332151 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:21.332226 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:21.332298 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 22:48:21.332877 (+ 579us) server_negotiation.cc:299] Negotiation successful
0806 22:48:21.332948 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":128,"threads_started":1}
I20260806 22:48:21.333701 9370 tablet_service.cc:1467] Processing CreateTablet for tablet bf5ce45c1617423094e79ec99b51cdab (DEFAULT_TABLE table=test-table [id=b44df97c4a944671a02f6f5077b85093]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:48:21.333755 9553 tablet_service.cc:1467] Processing CreateTablet for tablet bf5ce45c1617423094e79ec99b51cdab (DEFAULT_TABLE table=test-table [id=b44df97c4a944671a02f6f5077b85093]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:48:21.334097 9370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf5ce45c1617423094e79ec99b51cdab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:48:21.334117 9553 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf5ce45c1617423094e79ec99b51cdab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:48:21.338559 9665 tablet_bootstrap.cc:492] T bf5ce45c1617423094e79ec99b51cdab P 4756eb5819a543f4bbf440e26deb8284: Bootstrap starting.
I20260806 22:48:21.338660 9666 tablet_bootstrap.cc:492] T bf5ce45c1617423094e79ec99b51cdab P 53a97834ac9e48e8b3c5ad90490c9d25: Bootstrap starting.
I20260806 22:48:21.339591 9665 tablet_bootstrap.cc:654] T bf5ce45c1617423094e79ec99b51cdab P 4756eb5819a543f4bbf440e26deb8284: Neither blocks nor log segments found. Creating new log.
I20260806 22:48:21.339591 9666 tablet_bootstrap.cc:654] T bf5ce45c1617423094e79ec99b51cdab P 53a97834ac9e48e8b3c5ad90490c9d25: Neither blocks nor log segments found. Creating new log.
I20260806 22:48:21.340360 9665 log.cc:824] T bf5ce45c1617423094e79ec99b51cdab P 4756eb5819a543f4bbf440e26deb8284: Log is configured to *not* fsync() on all Append() calls
I20260806 22:48:21.340360 9666 log.cc:824] T bf5ce45c1617423094e79ec99b51cdab P 53a97834ac9e48e8b3c5ad90490c9d25: Log is configured to *not* fsync() on all Append() calls
I20260806 22:48:21.341317 9665 tablet_bootstrap.cc:492] T bf5ce45c1617423094e79ec99b51cdab P 4756eb5819a543f4bbf440e26deb8284: No bootstrap required, opened a new log
I20260806 22:48:21.341325 9666 tablet_bootstrap.cc:492] T bf5ce45c1617423094e79ec99b51cdab P 53a97834ac9e48e8b3c5ad90490c9d25: No bootstrap required, opened a new log
I20260806 22:48:21.341403 9665 ts_tablet_manager.cc:1397] T bf5ce45c1617423094e79ec99b51cdab P 4756eb5819a543f4bbf440e26deb8284: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260806 22:48:21.341401 9666 ts_tablet_manager.cc:1397] T bf5ce45c1617423094e79ec99b51cdab P 53a97834ac9e48e8b3c5ad90490c9d25: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:48:21.343705 9665 raft_consensus.cc:348] T bf5ce45c1617423094e79ec99b51cdab P 4756eb5819a543f4bbf440e26deb8284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4756eb5819a543f4bbf440e26deb8284" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 34433 } } peers { permanent_uuid: "53a97834ac9e48e8b3c5ad90490c9d25" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 45699 } } peers { permanent_uuid: "e70eeb6d81fd488b9c80b9c6de85eccc" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 41625 } }
I20260806 22:48:21.343705 9666 raft_consensus.cc:348] T bf5ce45c1617423094e79ec99b51cdab P 53a97834ac9e48e8b3c5ad90490c9d25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4756eb5819a543f4bbf440e26deb8284" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 34433 } } peers { permanent_uuid: "53a97834ac9e48e8b3c5ad90490c9d25" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 45699 } } peers { permanent_uuid: "e70eeb6d81fd488b9c80b9c6de85eccc" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 41625 } }
I20260806 22:48:21.343936 9665 raft_consensus.cc:374] T bf5ce45c1617423094e79ec99b51cdab P 4756eb5819a543f4bbf440e26deb8284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:48:21.343936 9666 raft_consensus.cc:374] T bf5ce45c1617423094e79ec99b51cdab P 53a97834ac9e48e8b3c5ad90490c9d25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:48:21.343986 9666 raft_consensus.cc:729] T bf5ce45c1617423094e79ec99b51cdab P 53a97834ac9e48e8b3c5ad90490c9d25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53a97834ac9e48e8b3c5ad90490c9d25, State: Initialized, Role: FOLLOWER
I20260806 22:48:21.343984 9665 raft_consensus.cc:729] T bf5ce45c1617423094e79ec99b51cdab P 4756eb5819a543f4bbf440e26deb8284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4756eb5819a543f4bbf440e26deb8284, State: Initialized, Role: FOLLOWER
I20260806 22:48:21.344131 9666 consensus_queue.cc:260] T bf5ce45c1617423094e79ec99b51cdab P 53a97834ac9e48e8b3c5ad90490c9d25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4756eb5819a543f4bbf440e26deb8284" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 34433 } } peers { permanent_uuid: "53a97834ac9e48e8b3c5ad90490c9d25" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 45699 } } peers { permanent_uuid: "e70eeb6d81fd488b9c80b9c6de85eccc" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 41625 } }
I20260806 22:48:21.344131 9665 consensus_queue.cc:260] T bf5ce45c1617423094e79ec99b51cdab P 4756eb5819a543f4bbf440e26deb8284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4756eb5819a543f4bbf440e26deb8284" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 34433 } } peers { permanent_uuid: "53a97834ac9e48e8b3c5ad90490c9d25" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 45699 } } peers { permanent_uuid: "e70eeb6d81fd488b9c80b9c6de85eccc" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 41625 } }
I20260806 22:48:21.344483 9665 ts_tablet_manager.cc:1428] T bf5ce45c1617423094e79ec99b51cdab P 4756eb5819a543f4bbf440e26deb8284: Time spent starting tablet: real 0.003s user 0.003s sys 0.001s
I20260806 22:48:21.344483 9666 ts_tablet_manager.cc:1428] T bf5ce45c1617423094e79ec99b51cdab P 53a97834ac9e48e8b3c5ad90490c9d25: Time spent starting tablet: real 0.003s user 0.001s sys 0.003s
I20260806 22:48:21.344511 9651 heartbeater.cc:503] Master 127.18.45.254:32823 was elected leader, sending a full tablet report...
I20260806 22:48:21.344638 9462 heartbeater.cc:503] Master 127.18.45.254:32823 was elected leader, sending a full tablet report...
W20260806 22:48:21.398001 9463 tablet.cc:2369] T bf5ce45c1617423094e79ec99b51cdab P 4756eb5819a543f4bbf440e26deb8284: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:48:21.489436 9270 tablet.cc:2369] T bf5ce45c1617423094e79ec99b51cdab P e70eeb6d81fd488b9c80b9c6de85eccc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:48:21.524138 9276 raft_consensus.cc:482] T bf5ce45c1617423094e79ec99b51cdab P e70eeb6d81fd488b9c80b9c6de85eccc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:48:21.524345 9276 raft_consensus.cc:504] T bf5ce45c1617423094e79ec99b51cdab P e70eeb6d81fd488b9c80b9c6de85eccc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4756eb5819a543f4bbf440e26deb8284" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 34433 } } peers { permanent_uuid: "53a97834ac9e48e8b3c5ad90490c9d25" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 45699 } } peers { permanent_uuid: "e70eeb6d81fd488b9c80b9c6de85eccc" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 41625 } }
I20260806 22:48:21.524866 9276 leader_election.cc:288] T bf5ce45c1617423094e79ec99b51cdab P e70eeb6d81fd488b9c80b9c6de85eccc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4756eb5819a543f4bbf440e26deb8284 (127.18.45.194:34433), 53a97834ac9e48e8b3c5ad90490c9d25 (127.18.45.195:45699)
I20260806 22:48:21.528501 9660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:21.525315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:49339 (local address 127.18.45.195:45699)
0806 22:48:21.525371 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 22:48:21.525373 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:48:21.525624 (+ 251us) server_negotiation.cc:407] Connection header received
0806 22:48:21.525664 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:48:21.525665 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:48:21.525678 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:48:21.525745 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:48:21.525938 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:21.526443 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:21.527639 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:21.528076 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:21.528164 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:21.528178 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0806 22:48:21.528342 (+ 164us) server_negotiation.cc:299] Negotiation successful
0806 22:48:21.528392 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 22:48:21.528503 9662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:21.525280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:36051 (local address 127.18.45.194:34433)
0806 22:48:21.525364 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0806 22:48:21.525367 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:48:21.525492 (+ 125us) server_negotiation.cc:407] Connection header received
0806 22:48:21.525544 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:48:21.525545 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:48:21.525565 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:48:21.525640 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:48:21.525887 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:21.526441 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:21.527608 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:21.528082 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:21.528190 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:21.528202 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0806 22:48:21.528366 (+ 164us) server_negotiation.cc:299] Negotiation successful
0806 22:48:21.528420 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 22:48:21.528517 9671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:21.525165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.195:45699 (local address 127.18.45.193:49339)
0806 22:48:21.525472 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:21.525485 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:48:21.525590 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:48:21.525759 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:48:21.525759 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:48:21.525762 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:48:21.525841 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:48:21.525844 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:48:21.526553 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:48:21.526554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:48:21.527512 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:48:21.527515 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:48:21.528167 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:48:21.528168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:48:21.528252 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:21.528258 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 22:48:21.528309 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 22:48:21.528394 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"mutex_wait_us":53,"thread_start_us":84,"threads_started":1}
I20260806 22:48:21.528532 9674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:21.525165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.194:34433 (local address 127.18.45.193:36051)
0806 22:48:21.525368 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:21.525384 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 22:48:21.525480 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:48:21.525655 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:48:21.525657 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:48:21.525660 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:48:21.525747 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:48:21.525750 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:48:21.526545 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:48:21.526546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:48:21.527506 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:48:21.527508 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:48:21.528200 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:48:21.528201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:48:21.528265 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:21.528268 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:48:21.528340 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 22:48:21.528392 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"mutex_wait_us":34,"thread_start_us":91,"threads_started":1}
I20260806 22:48:21.529002 9396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf5ce45c1617423094e79ec99b51cdab" candidate_uuid: "e70eeb6d81fd488b9c80b9c6de85eccc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4756eb5819a543f4bbf440e26deb8284" is_pre_election: true
I20260806 22:48:21.529052 9589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf5ce45c1617423094e79ec99b51cdab" candidate_uuid: "e70eeb6d81fd488b9c80b9c6de85eccc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53a97834ac9e48e8b3c5ad90490c9d25" is_pre_election: true
I20260806 22:48:21.529263 9396 raft_consensus.cc:2454] T bf5ce45c1617423094e79ec99b51cdab P 4756eb5819a543f4bbf440e26deb8284 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e70eeb6d81fd488b9c80b9c6de85eccc in term 0.
I20260806 22:48:21.529311 9589 raft_consensus.cc:2454] T bf5ce45c1617423094e79ec99b51cdab P 53a97834ac9e48e8b3c5ad90490c9d25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e70eeb6d81fd488b9c80b9c6de85eccc in term 0.
I20260806 22:48:21.529569 9121 leader_election.cc:302] T bf5ce45c1617423094e79ec99b51cdab P e70eeb6d81fd488b9c80b9c6de85eccc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4756eb5819a543f4bbf440e26deb8284, e70eeb6d81fd488b9c80b9c6de85eccc; no voters:
I20260806 22:48:21.529776 9276 raft_consensus.cc:2781] T bf5ce45c1617423094e79ec99b51cdab P e70eeb6d81fd488b9c80b9c6de85eccc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:48:21.529889 9276 raft_consensus.cc:482] T bf5ce45c1617423094e79ec99b51cdab P e70eeb6d81fd488b9c80b9c6de85eccc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:48:21.529915 9276 raft_consensus.cc:3037] T bf5ce45c1617423094e79ec99b51cdab P e70eeb6d81fd488b9c80b9c6de85eccc [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:48:21.530833 9276 raft_consensus.cc:504] T bf5ce45c1617423094e79ec99b51cdab P e70eeb6d81fd488b9c80b9c6de85eccc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4756eb5819a543f4bbf440e26deb8284" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 34433 } } peers { permanent_uuid: "53a97834ac9e48e8b3c5ad90490c9d25" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 45699 } } peers { permanent_uuid: "e70eeb6d81fd488b9c80b9c6de85eccc" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 41625 } }
I20260806 22:48:21.531041 9276 leader_election.cc:288] T bf5ce45c1617423094e79ec99b51cdab P e70eeb6d81fd488b9c80b9c6de85eccc [CANDIDATE]: Term 1 election: Requested vote from peers 4756eb5819a543f4bbf440e26deb8284 (127.18.45.194:34433), 53a97834ac9e48e8b3c5ad90490c9d25 (127.18.45.195:45699)
I20260806 22:48:21.531255 9396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf5ce45c1617423094e79ec99b51cdab" candidate_uuid: "e70eeb6d81fd488b9c80b9c6de85eccc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4756eb5819a543f4bbf440e26deb8284"
I20260806 22:48:21.531266 9589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf5ce45c1617423094e79ec99b51cdab" candidate_uuid: "e70eeb6d81fd488b9c80b9c6de85eccc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53a97834ac9e48e8b3c5ad90490c9d25"
I20260806 22:48:21.531374 9396 raft_consensus.cc:3037] T bf5ce45c1617423094e79ec99b51cdab P 4756eb5819a543f4bbf440e26deb8284 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:48:21.531369 9589 raft_consensus.cc:3037] T bf5ce45c1617423094e79ec99b51cdab P 53a97834ac9e48e8b3c5ad90490c9d25 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:48:21.532261 9589 raft_consensus.cc:2454] T bf5ce45c1617423094e79ec99b51cdab P 53a97834ac9e48e8b3c5ad90490c9d25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e70eeb6d81fd488b9c80b9c6de85eccc in term 1.
I20260806 22:48:21.532390 9396 raft_consensus.cc:2454] T bf5ce45c1617423094e79ec99b51cdab P 4756eb5819a543f4bbf440e26deb8284 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e70eeb6d81fd488b9c80b9c6de85eccc in term 1.
I20260806 22:48:21.532456 9119 leader_election.cc:302] T bf5ce45c1617423094e79ec99b51cdab P e70eeb6d81fd488b9c80b9c6de85eccc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53a97834ac9e48e8b3c5ad90490c9d25, e70eeb6d81fd488b9c80b9c6de85eccc; no voters:
I20260806 22:48:21.532575 9276 raft_consensus.cc:2781] T bf5ce45c1617423094e79ec99b51cdab P e70eeb6d81fd488b9c80b9c6de85eccc [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:48:21.532759 9276 raft_consensus.cc:686] T bf5ce45c1617423094e79ec99b51cdab P e70eeb6d81fd488b9c80b9c6de85eccc [term 1 LEADER]: Becoming Leader. State: Replica: e70eeb6d81fd488b9c80b9c6de85eccc, State: Running, Role: LEADER
I20260806 22:48:21.532845 9276 consensus_queue.cc:237] T bf5ce45c1617423094e79ec99b51cdab P e70eeb6d81fd488b9c80b9c6de85eccc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4756eb5819a543f4bbf440e26deb8284" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 34433 } } peers { permanent_uuid: "53a97834ac9e48e8b3c5ad90490c9d25" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 45699 } } peers { permanent_uuid: "e70eeb6d81fd488b9c80b9c6de85eccc" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 41625 } }
I20260806 22:48:21.533879 9012 catalog_manager.cc:5495] T bf5ce45c1617423094e79ec99b51cdab P e70eeb6d81fd488b9c80b9c6de85eccc reported cstate change: term changed from 0 to 1, leader changed from <none> to e70eeb6d81fd488b9c80b9c6de85eccc (127.18.45.193). New cstate: current_term: 1 leader_uuid: "e70eeb6d81fd488b9c80b9c6de85eccc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4756eb5819a543f4bbf440e26deb8284" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 34433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53a97834ac9e48e8b3c5ad90490c9d25" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 45699 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e70eeb6d81fd488b9c80b9c6de85eccc" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 41625 } health_report { overall_health: HEALTHY } } }
W20260806 22:48:21.544730 9652 tablet.cc:2369] T bf5ce45c1617423094e79ec99b51cdab P 53a97834ac9e48e8b3c5ad90490c9d25: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:48:21.568574 9679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:21.565230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42036 (local address 127.18.45.193:41625)
0806 22:48:21.565484 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0806 22:48:21.565487 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:48:21.565497 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:48:21.565557 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:48:21.565557 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:48:21.565585 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:48:21.565716 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:48:21.565889 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:21.566510 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:21.567330 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:21.567545 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:21.567621 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:21.568114 (+ 493us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:48:21.568244 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:48:21.568340 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 22:48:21.568387 (+ 47us) server_negotiation.cc:299] Negotiation successful
0806 22:48:21.568413 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":132,"threads_started":1}
I20260806 22:48:21.570766 9589 raft_consensus.cc:1261] T bf5ce45c1617423094e79ec99b51cdab P 53a97834ac9e48e8b3c5ad90490c9d25 [term 1 FOLLOWER]: Refusing update from remote peer e70eeb6d81fd488b9c80b9c6de85eccc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:48:21.570768 9396 raft_consensus.cc:1261] T bf5ce45c1617423094e79ec99b51cdab P 4756eb5819a543f4bbf440e26deb8284 [term 1 FOLLOWER]: Refusing update from remote peer e70eeb6d81fd488b9c80b9c6de85eccc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:48:21.571231 9675 consensus_queue.cc:1035] T bf5ce45c1617423094e79ec99b51cdab P e70eeb6d81fd488b9c80b9c6de85eccc [LEADER]: Connected to new peer: Peer: permanent_uuid: "4756eb5819a543f4bbf440e26deb8284" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 34433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:48:21.571381 9276 consensus_queue.cc:1035] T bf5ce45c1617423094e79ec99b51cdab P e70eeb6d81fd488b9c80b9c6de85eccc [LEADER]: Connected to new peer: Peer: permanent_uuid: "53a97834ac9e48e8b3c5ad90490c9d25" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 45699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:48:21.574402 9680 mvcc.cc:205] Tried to move back new op lower bound from 7315687430430355456 to 7315687430279573504. Current Snapshot: MvccSnapshot[applied={T|T < 7315687430430355456}]
I20260806 22:48:21.575160 9682 mvcc.cc:205] Tried to move back new op lower bound from 7315687430430355456 to 7315687430279573504. Current Snapshot: MvccSnapshot[applied={T|T < 7315687430430355456}]
I20260806 22:48:21.575286 9681 mvcc.cc:205] Tried to move back new op lower bound from 7315687430430355456 to 7315687430279573504. Current Snapshot: MvccSnapshot[applied={T|T < 7315687430430355456}]
I20260806 22:48:21.577569 9012 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35258:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:48:21.577688 9012 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35258:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:48:21.578672 9012 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419: Sending DeleteTablet for 3 replicas of tablet bf5ce45c1617423094e79ec99b51cdab
I20260806 22:48:21.579069 9176 tablet_service.cc:1514] Processing DeleteTablet for tablet bf5ce45c1617423094e79ec99b51cdab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:48:21 UTC) from {username='kudu', principal='kudu/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:42032
I20260806 22:48:21.579133 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 9070
I20260806 22:48:21.582242 9370 tablet_service.cc:1514] Processing DeleteTablet for tablet bf5ce45c1617423094e79ec99b51cdab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:48:21 UTC) from {username='kudu', principal='kudu/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:50160
I20260806 22:48:21.582573 9553 tablet_service.cc:1514] Processing DeleteTablet for tablet bf5ce45c1617423094e79ec99b51cdab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:48:21 UTC) from {username='kudu', principal='kudu/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:38070
I20260806 22:48:21.582952 9696 tablet_replica.cc:330] T bf5ce45c1617423094e79ec99b51cdab P 4756eb5819a543f4bbf440e26deb8284: stopping tablet replica
I20260806 22:48:21.582950 9698 tablet_replica.cc:330] T bf5ce45c1617423094e79ec99b51cdab P 53a97834ac9e48e8b3c5ad90490c9d25: stopping tablet replica
I20260806 22:48:21.583102 9696 raft_consensus.cc:2229] T bf5ce45c1617423094e79ec99b51cdab P 4756eb5819a543f4bbf440e26deb8284 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:48:21.583209 9698 raft_consensus.cc:2229] T bf5ce45c1617423094e79ec99b51cdab P 53a97834ac9e48e8b3c5ad90490c9d25 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:48:21.583241 9696 raft_consensus.cc:2258] T bf5ce45c1617423094e79ec99b51cdab P 4756eb5819a543f4bbf440e26deb8284 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:48:21.583395 9698 raft_consensus.cc:2258] T bf5ce45c1617423094e79ec99b51cdab P 53a97834ac9e48e8b3c5ad90490c9d25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:48:21.586845 9696 ts_tablet_manager.cc:1905] T bf5ce45c1617423094e79ec99b51cdab P 4756eb5819a543f4bbf440e26deb8284: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:48:21.586849 9698 ts_tablet_manager.cc:1905] T bf5ce45c1617423094e79ec99b51cdab P 53a97834ac9e48e8b3c5ad90490c9d25: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:48:21.589255 9698 ts_tablet_manager.cc:1918] T bf5ce45c1617423094e79ec99b51cdab P 53a97834ac9e48e8b3c5ad90490c9d25: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:48:21.589349 9698 log.cc:1196] T bf5ce45c1617423094e79ec99b51cdab P 53a97834ac9e48e8b3c5ad90490c9d25: Deleting WAL directory at /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-2/wal/wals/bf5ce45c1617423094e79ec99b51cdab
I20260806 22:48:21.589303 9696 ts_tablet_manager.cc:1918] T bf5ce45c1617423094e79ec99b51cdab P 4756eb5819a543f4bbf440e26deb8284: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:48:21.589570 9696 log.cc:1196] T bf5ce45c1617423094e79ec99b51cdab P 4756eb5819a543f4bbf440e26deb8284: Deleting WAL directory at /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-1/wal/wals/bf5ce45c1617423094e79ec99b51cdab
I20260806 22:48:21.589721 9698 ts_tablet_manager.cc:1939] T bf5ce45c1617423094e79ec99b51cdab P 53a97834ac9e48e8b3c5ad90490c9d25: Deleting consensus metadata
I20260806 22:48:21.589955 9696 ts_tablet_manager.cc:1939] T bf5ce45c1617423094e79ec99b51cdab P 4756eb5819a543f4bbf440e26deb8284: Deleting consensus metadata
I20260806 22:48:21.590337 8997 catalog_manager.cc:4841] TS 53a97834ac9e48e8b3c5ad90490c9d25 (127.18.45.195:45699): tablet bf5ce45c1617423094e79ec99b51cdab (table test-table [id=b44df97c4a944671a02f6f5077b85093]) successfully deleted
I20260806 22:48:21.590466 8998 catalog_manager.cc:4841] TS 4756eb5819a543f4bbf440e26deb8284 (127.18.45.194:34433): tablet bf5ce45c1617423094e79ec99b51cdab (table test-table [id=b44df97c4a944671a02f6f5077b85093]) successfully deleted
W20260806 22:48:21.598657 8995 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.45.193:41625 (error 108)
I20260806 22:48:21.598991 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 9273
I20260806 22:48:21.612195 9658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:21.611839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.193:41625 (local address 127.0.0.1:42040)
0806 22:48:21.612028 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:21.612119 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.193:41625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148}
W20260806 22:48:21.612466 8995 catalog_manager.cc:4584] TS e70eeb6d81fd488b9c80b9c6de85eccc (127.18.45.193:41625): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bf5ce45c1617423094e79ec99b51cdab: Network error: Client connection negotiation failed: client connection to 127.18.45.193:41625: connect: Connection refused (error 111)
I20260806 22:48:21.618001 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 9465
I20260806 22:48:21.630935 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 8974
I20260806 22:48:41.646179 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:32823
--webserver_interface=127.18.45.254
--webserver_port=41899
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:41985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.45.254:32823
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:48:41.743362 9700 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:48:41.743628 9700 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:48:41.743659 9700 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:48:41.744961 9700 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 22:48:41.745008 9700 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 22:48:41.745021 9700 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 22:48:41.745033 9700 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:48:41.745046 9700 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:48:41.745059 9700 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:48:41.745072 9700 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:48:41.745085 9700 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:48:41.746665 9700 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:41985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.18.45.254:32823
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:32823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.45.254
--webserver_port=41899
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9700
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:48:41.746939 9700 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:48:41.747232 9700 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 22:48:41.753437 9700 server_base.cc:1034] running on GCE node
W20260806 22:48:41.753384 9710 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:48:41.753356 9709 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:48:41.753357 9712 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:48:41.754141 9700 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:48:41.754536 9700 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:48:41.755748 9700 hybrid_clock.cc:648] HybridClock initialized: now 1786056521755727 us; error 39 us; skew 500 ppm
Aug 06 22:48:41 dist-test-slave-h3ft krb5kdc[8237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056521, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:48:41.763602 9700 init.cc:377] Logged in from keytab as kudu/127.18.45.254@KRBTEST.COM (short username kudu)
I20260806 22:48:41.764236 9700 webserver.cc:460] Webserver started at http://127.18.45.254:41899/ using document root <none> and password file <none>
I20260806 22:48:41.764547 9700 fs_manager.cc:359] Metadata directory not provided
I20260806 22:48:41.764601 9700 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:48:41.766286 9700 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260806 22:48:41.767272 9720 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:48:41.767596 9700 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:48:41.767724 9700 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/master-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/master-0/wal
uuid: "2817a20f1830402dbc70359306e75419"
format_stamp: "Formatted at 2026-08-06 22:47:59 on dist-test-slave-h3ft"
server_key: "2e9805f58a1ffd833fa20b15e90cddc4"
server_key_iv: "95356632211c9ff1b9aac382ea71a14c"
server_key_version: "encryptionkey@0"
I20260806 22:48:41.768242 9700 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:48:41.781787 9700 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:48:41.784657 9700 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:48:41.784865 9700 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:48:41.790417 9700 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.254:32823
I20260806 22:48:41.790536 9792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.254:32823 every 8 connection(s)
I20260806 22:48:41.790993 9700 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
I20260806 22:48:41.793373 9793 sys_catalog.cc:263] Verifying existing consensus state
I20260806 22:48:41.794188 9793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419: Bootstrap starting.
I20260806 22:48:41.794901 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 9700
I20260806 22:48:41.795310 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.193:41625
--local_ip_for_outbound_sockets=127.18.45.193
--tserver_master_addrs=127.18.45.254:32823
--webserver_port=38949
--webserver_interface=127.18.45.193
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab
--principal=kudu/127.18.45.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:41985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:48:41.807600 9793 log.cc:824] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419: Log is configured to *not* fsync() on all Append() calls
I20260806 22:48:41.811344 9793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:48:41.811770 9793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419: Bootstrap complete.
I20260806 22:48:41.814936 9793 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2817a20f1830402dbc70359306e75419" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 32823 } }
I20260806 22:48:41.815339 9793 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2817a20f1830402dbc70359306e75419, State: Initialized, Role: FOLLOWER
I20260806 22:48:41.815599 9793 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2817a20f1830402dbc70359306e75419" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 32823 } }
I20260806 22:48:41.815673 9793 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:48:41.815740 9793 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:48:41.815799 9793 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419 [term 2 FOLLOWER]: Advancing to term 3
I20260806 22:48:41.817142 9793 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2817a20f1830402dbc70359306e75419" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 32823 } }
I20260806 22:48:41.817281 9793 leader_election.cc:302] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2817a20f1830402dbc70359306e75419; no voters:
I20260806 22:48:41.817559 9793 leader_election.cc:288] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 22:48:41.817622 9797 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 22:48:41.817801 9797 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419 [term 3 LEADER]: Becoming Leader. State: Replica: 2817a20f1830402dbc70359306e75419, State: Running, Role: LEADER
I20260806 22:48:41.817922 9797 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2817a20f1830402dbc70359306e75419" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 32823 } }
I20260806 22:48:41.817978 9793 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:48:41.818281 9798 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "2817a20f1830402dbc70359306e75419" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2817a20f1830402dbc70359306e75419" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 32823 } } }
I20260806 22:48:41.818392 9798 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419 [sys.catalog]: This master's current role is: LEADER
I20260806 22:48:41.818274 9797 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2817a20f1830402dbc70359306e75419. Latest consensus state: current_term: 3 leader_uuid: "2817a20f1830402dbc70359306e75419" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2817a20f1830402dbc70359306e75419" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 32823 } } }
I20260806 22:48:41.818641 9797 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419 [sys.catalog]: This master's current role is: LEADER
I20260806 22:48:41.819032 9801 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:48:41.820104 9801 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:48:41.820220 9801 catalog_manager.cc:1259] Loaded cluster ID: 1b4274433ca54a8183f72fe367b61dba
I20260806 22:48:41.820252 9801 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:48:41.821502 9801 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:48:41.821713 9801 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419: Loaded TSK: 0
I20260806 22:48:41.821762 9801 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2817a20f1830402dbc70359306e75419: Loaded TSK: 1
F20260806 22:48:41.822006 9801 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f3c6bd589ad google::LogMessage::Fail() at ??:0
@ 0x7f3c6bd5b88a google::LogMessage::SendToLog() at ??:0
@ 0x7f3c6bd584de google::LogMessage::Flush() at ??:0
@ 0x7f3c6bd59a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f3c6e4c5104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f3c6e4c2d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f3c73cb21cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f3c73cb248e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f3c73cb2594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f3c73c78c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f3c73c82ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f3c6dc260d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f3c6dc1b954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f3c6ea7f6db start_thread at ??:0
@ 0x7f3c6a85661f clone at ??:0
W20260806 22:48:41.916796 9795 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:48:41.917047 9795 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:48:41.917071 9795 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:48:41.918292 9795 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:48:41.918339 9795 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:48:41.918366 9795 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.193
I20260806 22:48:41.920011 9795 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:41985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab
--principal=kudu/127.18.45.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.193:41625
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.45.193
--webserver_port=38949
--tserver_master_addrs=127.18.45.254:32823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9795
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.193
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:48:41.920293 9795 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:48:41.920588 9795 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:48:41.926808 9853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:48:41.926811 9849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:48:41.926826 9850 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:48:41.927129 9795 server_base.cc:1034] running on GCE node
I20260806 22:48:41.927511 9795 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:48:41.927817 9795 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:48:41.928987 9795 hybrid_clock.cc:648] HybridClock initialized: now 1786056521928963 us; error 37 us; skew 500 ppm
Aug 06 22:48:41 dist-test-slave-h3ft krb5kdc[8237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056521, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:48:41.932001 9795 init.cc:377] Logged in from keytab as kudu/127.18.45.193@KRBTEST.COM (short username kudu)
I20260806 22:48:41.932654 9795 webserver.cc:460] Webserver started at http://127.18.45.193:38949/ using document root <none> and password file <none>
I20260806 22:48:41.932933 9795 fs_manager.cc:359] Metadata directory not provided
I20260806 22:48:41.932984 9795 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:48:41.934607 9795 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 22:48:41.935490 9864 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:48:41.935704 9795 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 22:48:41.935843 9795 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-0/wal
uuid: "e70eeb6d81fd488b9c80b9c6de85eccc"
format_stamp: "Formatted at 2026-08-06 22:47:59 on dist-test-slave-h3ft"
server_key: "cd42bc14687e788551e73e3f92592154"
server_key_iv: "2e1e4ec8c02c735da89f96ce03bed992"
server_key_version: "encryptionkey@0"
I20260806 22:48:41.936441 9795 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:48:41.950850 9795 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:48:41.954068 9795 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:48:41.954259 9795 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:48:41.954600 9795 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:48:41.955397 9873 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 22:48:41.957389 9795 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 22:48:41.957474 9795 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260806 22:48:41.957520 9795 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 22:48:41.958634 9795 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 22:48:41.958688 9795 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260806 22:48:41.958902 9873 tablet_bootstrap.cc:492] T bf5ce45c1617423094e79ec99b51cdab P e70eeb6d81fd488b9c80b9c6de85eccc: Bootstrap starting.
I20260806 22:48:41.968789 9795 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.193:41625
I20260806 22:48:41.968879 10018 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.193:41625 every 8 connection(s)
I20260806 22:48:41.969441 9795 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
I20260806 22:48:41.974176 9873 log.cc:824] T bf5ce45c1617423094e79ec99b51cdab P e70eeb6d81fd488b9c80b9c6de85eccc: Log is configured to *not* fsync() on all Append() calls
I20260806 22:48:41.974370 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 9795
I20260806 22:48:41.974722 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.194:34433
--local_ip_for_outbound_sockets=127.18.45.194
--tserver_master_addrs=127.18.45.254:32823
--webserver_port=38907
--webserver_interface=127.18.45.194
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab
--principal=kudu/127.18.45.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:41985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:48:41.977746 9873 tablet_bootstrap.cc:492] T bf5ce45c1617423094e79ec99b51cdab P e70eeb6d81fd488b9c80b9c6de85eccc: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 22:48:41.978083 9873 tablet_bootstrap.cc:492] T bf5ce45c1617423094e79ec99b51cdab P e70eeb6d81fd488b9c80b9c6de85eccc: Bootstrap complete.
I20260806 22:48:41.979327 9873 ts_tablet_manager.cc:1397] T bf5ce45c1617423094e79ec99b51cdab P e70eeb6d81fd488b9c80b9c6de85eccc: Time spent bootstrapping tablet: real 0.021s user 0.002s sys 0.017s
I20260806 22:48:41.981319 9873 raft_consensus.cc:348] T bf5ce45c1617423094e79ec99b51cdab P e70eeb6d81fd488b9c80b9c6de85eccc [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4756eb5819a543f4bbf440e26deb8284" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 34433 } } peers { permanent_uuid: "53a97834ac9e48e8b3c5ad90490c9d25" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 45699 } } peers { permanent_uuid: "e70eeb6d81fd488b9c80b9c6de85eccc" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 41625 } }
I20260806 22:48:41.981757 9873 raft_consensus.cc:729] T bf5ce45c1617423094e79ec99b51cdab P e70eeb6d81fd488b9c80b9c6de85eccc [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e70eeb6d81fd488b9c80b9c6de85eccc, State: Initialized, Role: FOLLOWER
I20260806 22:48:41.981979 9873 consensus_queue.cc:260] T bf5ce45c1617423094e79ec99b51cdab P e70eeb6d81fd488b9c80b9c6de85eccc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4756eb5819a543f4bbf440e26deb8284" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 34433 } } peers { permanent_uuid: "53a97834ac9e48e8b3c5ad90490c9d25" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 45699 } } peers { permanent_uuid: "e70eeb6d81fd488b9c80b9c6de85eccc" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 41625 } }
I20260806 22:48:41.982337 9873 ts_tablet_manager.cc:1428] T bf5ce45c1617423094e79ec99b51cdab P e70eeb6d81fd488b9c80b9c6de85eccc: Time spent starting tablet: real 0.003s user 0.001s sys 0.003s
W20260806 22:48:42.061902 10022 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:48:41.970445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:32823 (local address 127.18.45.193:58927)
0806 22:48:41.970751 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:41.970815 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0806 22:48:41.971792 (+ 977us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:48:42.061680 (+ 89888us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:32823: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":135,"threads_started":1}
W20260806 22:48:42.062291 10019 heartbeater.cc:644] Failed to heartbeat to 127.18.45.254:32823 (0 consecutive failures): Network error: Failed to ping master at 127.18.45.254:32823: Client connection negotiation failed: client connection to 127.18.45.254:32823: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 22:48:42.062697 10022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:42.062556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:32823 (local address 127.18.45.193:50755)
0806 22:48:42.062619 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:42.062675 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260806 22:48:42.063237 10022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:42.063127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:32823 (local address 127.18.45.193:52923)
0806 22:48:42.063178 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:42.063206 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260806 22:48:42.063362 10019 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 22:48:42.076996 10024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:48:42.077288 10024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:48:42.077333 10024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:48:42.078630 10024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:48:42.078702 10024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:48:42.078750 10024 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.194
I20260806 22:48:42.080524 10024 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:41985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab
--principal=kudu/127.18.45.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.194:34433
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.45.194
--webserver_port=38907
--tserver_master_addrs=127.18.45.254:32823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10024
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.194
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:48:42.080853 10024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:48:42.081156 10024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:48:42.087268 10032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:48:42.087335 10031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:48:42.087395 10024 server_base.cc:1034] running on GCE node
W20260806 22:48:42.087301 10035 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:48:42.087670 10024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:48:42.087975 10024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:48:42.089150 10024 hybrid_clock.cc:648] HybridClock initialized: now 1786056522089135 us; error 58 us; skew 500 ppm
Aug 06 22:48:42 dist-test-slave-h3ft krb5kdc[8237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056522, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:48:42.092257 10024 init.cc:377] Logged in from keytab as kudu/127.18.45.194@KRBTEST.COM (short username kudu)
I20260806 22:48:42.092808 10024 webserver.cc:460] Webserver started at http://127.18.45.194:38907/ using document root <none> and password file <none>
I20260806 22:48:42.093119 10024 fs_manager.cc:359] Metadata directory not provided
I20260806 22:48:42.093199 10024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:48:42.095072 10024 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 22:48:42.095952 10046 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:48:42.096216 10024 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:48:42.096342 10024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-1/wal
uuid: "4756eb5819a543f4bbf440e26deb8284"
format_stamp: "Formatted at 2026-08-06 22:47:59 on dist-test-slave-h3ft"
server_key: "0889662080420627d084d3b1538d3ef9"
server_key_iv: "8851ad8f354a9ce5e6d2bd0c3067b1b7"
server_key_version: "encryptionkey@0"
I20260806 22:48:42.096810 10024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:48:42.107266 10024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:48:42.109817 10024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:48:42.110093 10024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:48:42.110400 10024 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:48:42.110785 10024 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:48:42.110843 10024 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:48:42.110901 10024 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:48:42.110934 10024 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:48:42.118937 10024 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.194:34433
I20260806 22:48:42.119056 10195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.194:34433 every 8 connection(s)
I20260806 22:48:42.119498 10024 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
I20260806 22:48:42.120987 10199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:42.120465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:32823 (local address 127.18.45.194:57845)
0806 22:48:42.120663 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:42.120760 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":95,"threads_started":1}
W20260806 22:48:42.121533 10196 heartbeater.cc:644] Failed to heartbeat to 127.18.45.254:32823 (0 consecutive failures): Network error: Failed to ping master at 127.18.45.254:32823: Client connection negotiation failed: client connection to 127.18.45.254:32823: connect: Connection refused (error 111)
I20260806 22:48:42.121869 10199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:42.121751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:32823 (local address 127.18.45.194:51463)
0806 22:48:42.121820 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:42.121850 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260806 22:48:42.122193 10199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:42.122105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:32823 (local address 127.18.45.194:60795)
0806 22:48:42.122153 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:42.122177 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260806 22:48:42.122329 10196 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 22:48:42.125010 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 10024
I20260806 22:48:42.125388 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.195:45699
--local_ip_for_outbound_sockets=127.18.45.195
--tserver_master_addrs=127.18.45.254:32823
--webserver_port=42291
--webserver_interface=127.18.45.195
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab
--principal=kudu/127.18.45.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:41985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:48:42.229112 10203 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:48:42.229559 10203 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:48:42.229626 10203 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:48:42.232023 10203 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:48:42.232106 10203 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:48:42.232158 10203 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.195
I20260806 22:48:42.234023 10203 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:41985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab
--principal=kudu/127.18.45.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.195:45699
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.45.195
--webserver_port=42291
--tserver_master_addrs=127.18.45.254:32823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10203
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.195
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:48:42.234392 10203 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:48:42.234759 10203 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:48:42.241891 10213 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:48:42.241912 10212 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:48:42.241914 10215 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:48:42.242126 10203 server_base.cc:1034] running on GCE node
I20260806 22:48:42.242498 10203 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:48:42.242832 10203 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:48:42.244107 10203 hybrid_clock.cc:648] HybridClock initialized: now 1786056522244082 us; error 40 us; skew 500 ppm
Aug 06 22:48:42 dist-test-slave-h3ft krb5kdc[8237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056522, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:48:42.250597 10203 init.cc:377] Logged in from keytab as kudu/127.18.45.195@KRBTEST.COM (short username kudu)
I20260806 22:48:42.251365 10203 webserver.cc:460] Webserver started at http://127.18.45.195:42291/ using document root <none> and password file <none>
I20260806 22:48:42.251789 10203 fs_manager.cc:359] Metadata directory not provided
I20260806 22:48:42.251996 10203 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:48:42.254176 10203 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:48:42.255105 10223 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:48:42.255352 10203 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:48:42.255472 10203 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-2/wal
uuid: "53a97834ac9e48e8b3c5ad90490c9d25"
format_stamp: "Formatted at 2026-08-06 22:48:00 on dist-test-slave-h3ft"
server_key: "8b3fc2654f66064554342d0522709aaf"
server_key_iv: "9681cd6fa5631a2d288a7784fd80f65b"
server_key_version: "encryptionkey@0"
I20260806 22:48:42.256120 10203 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:48:42.275530 10203 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:48:42.278438 10203 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:48:42.278635 10203 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:48:42.278973 10203 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:48:42.279405 10203 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:48:42.279465 10203 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:48:42.279521 10203 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:48:42.279549 10203 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:48:42.287108 10203 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.195:45699
I20260806 22:48:42.287184 10386 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.195:45699 every 8 connection(s)
I20260806 22:48:42.287648 10203 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
I20260806 22:48:42.289233 10390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:42.288504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:32823 (local address 127.18.45.195:42285)
0806 22:48:42.288690 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:42.288788 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":84,"threads_started":1}
W20260806 22:48:42.289528 10387 heartbeater.cc:644] Failed to heartbeat to 127.18.45.254:32823 (0 consecutive failures): Network error: Failed to ping master at 127.18.45.254:32823: Client connection negotiation failed: client connection to 127.18.45.254:32823: connect: Connection refused (error 111)
I20260806 22:48:42.289831 10390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:42.289735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:32823 (local address 127.18.45.195:49055)
0806 22:48:42.289782 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:42.289811 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 22:48:42.290165 10390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:42.290109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:32823 (local address 127.18.45.195:52463)
0806 22:48:42.290131 (+ 22us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:42.290150 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260806 22:48:42.290282 10387 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 22:48:42.293445 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 10203
I20260806 22:48:43.064275 10392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:43.063775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:32823 (local address 127.18.45.193:59663)
0806 22:48:43.064132 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:43.064178 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":87,"threads_started":1}
I20260806 22:48:43.123183 10393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:43.122806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:32823 (local address 127.18.45.194:57595)
0806 22:48:43.123036 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:43.123095 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":99,"threads_started":1}
I20260806 22:48:43.155308 10394 raft_consensus.cc:482] T bf5ce45c1617423094e79ec99b51cdab P e70eeb6d81fd488b9c80b9c6de85eccc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:48:43.155579 10394 raft_consensus.cc:504] T bf5ce45c1617423094e79ec99b51cdab P e70eeb6d81fd488b9c80b9c6de85eccc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4756eb5819a543f4bbf440e26deb8284" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 34433 } } peers { permanent_uuid: "53a97834ac9e48e8b3c5ad90490c9d25" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 45699 } } peers { permanent_uuid: "e70eeb6d81fd488b9c80b9c6de85eccc" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 41625 } }
I20260806 22:48:43.156065 10394 leader_election.cc:288] T bf5ce45c1617423094e79ec99b51cdab P e70eeb6d81fd488b9c80b9c6de85eccc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4756eb5819a543f4bbf440e26deb8284 (127.18.45.194:34433), 53a97834ac9e48e8b3c5ad90490c9d25 (127.18.45.195:45699)
Aug 06 22:48:43 dist-test-slave-h3ft krb5kdc[8237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056521, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.193@KRBTEST.COM for kudu/127.18.45.195@KRBTEST.COM
Aug 06 22:48:43 dist-test-slave-h3ft krb5kdc[8237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056521, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.193@KRBTEST.COM for kudu/127.18.45.194@KRBTEST.COM
I20260806 22:48:43.170176 10398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:43.156518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:38717 (local address 127.18.45.194:34433)
0806 22:48:43.156719 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0806 22:48:43.156732 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 22:48:43.156769 (+ 37us) server_negotiation.cc:407] Connection header received
0806 22:48:43.157337 (+ 568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:48:43.157340 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:48:43.157401 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:48:43.157559 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:48:43.158493 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:43.160027 (+ 1534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:43.161808 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:43.162484 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:43.162609 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:43.166194 (+ 3585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:48:43.166198 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:48:43.166199 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:48:43.166212 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:48:43.168225 (+ 2013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:48:43.168667 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:48:43.168671 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:48:43.168671 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:48:43.168778 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:48:43.169082 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:48:43.169084 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:48:43.169084 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:48:43.169468 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:48:43.169579 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 22:48:43.169883 (+ 304us) server_negotiation.cc:299] Negotiation successful
0806 22:48:43.170036 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":87,"threads_started":1}
I20260806 22:48:43.170161 10392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:43.156311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.195:45699 (local address 127.18.45.193:51881)
0806 22:48:43.156396 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:43.156414 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 22:48:43.156488 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:48:43.157636 (+ 1148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:48:43.157639 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:48:43.158102 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:48:43.158336 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:48:43.158343 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:48:43.160162 (+ 1819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:48:43.160163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:48:43.161684 (+ 1521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:48:43.161688 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:48:43.162607 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:48:43.162609 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:48:43.162767 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:43.163538 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:48:43.163543 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:48:43.165695 (+ 2152us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:48:43.168232 (+ 2537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:48:43.168240 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:48:43.168242 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:48:43.168579 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:48:43.168960 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:48:43.168961 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:48:43.168962 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:48:43.169055 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:48:43.169556 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:48:43.169564 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:48:43.169706 (+ 142us) client_negotiation.cc:769] Sending connection context
0806 22:48:43.169829 (+ 123us) client_negotiation.cc:240] Negotiation successful
0806 22:48:43.170049 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61,"mutex_wait_us":107}
I20260806 22:48:43.170190 10399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:43.156518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:51881 (local address 127.18.45.195:45699)
0806 22:48:43.156745 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0806 22:48:43.156768 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0806 22:48:43.156793 (+ 25us) server_negotiation.cc:407] Connection header received
0806 22:48:43.157337 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:48:43.157340 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:48:43.157401 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:48:43.157610 (+ 209us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:48:43.158521 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:43.160039 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:43.161808 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:43.162482 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:43.162610 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:43.165822 (+ 3212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:48:43.165829 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:48:43.165829 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:48:43.165843 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:48:43.168122 (+ 2279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:48:43.168732 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:48:43.168735 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:48:43.168736 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:48:43.168880 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:48:43.169121 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:48:43.169122 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:48:43.169123 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:48:43.169477 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:48:43.169579 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 22:48:43.169896 (+ 317us) server_negotiation.cc:299] Negotiation successful
0806 22:48:43.170042 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":84,"threads_started":1}
I20260806 22:48:43.170289 10396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:43.156311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.194:34433 (local address 127.18.45.193:38717)
0806 22:48:43.156530 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:43.156539 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:48:43.156586 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:48:43.157637 (+ 1051us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:48:43.157640 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:48:43.158102 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:48:43.158335 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:48:43.158343 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:48:43.160179 (+ 1836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:48:43.160180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:48:43.161684 (+ 1504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:48:43.161688 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:48:43.162609 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:48:43.162610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:48:43.162766 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:43.163521 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:48:43.163533 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:48:43.166080 (+ 2547us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:48:43.168291 (+ 2211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:48:43.168292 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:48:43.168293 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:48:43.168579 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:48:43.168853 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:48:43.168853 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:48:43.168854 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:48:43.169000 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:48:43.169568 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:48:43.169569 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:48:43.169706 (+ 137us) client_negotiation.cc:769] Sending connection context
0806 22:48:43.169822 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 22:48:43.170138 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"mutex_wait_us":262,"thread_start_us":110,"threads_started":1}
W20260806 22:48:43.171653 9868 leader_election.cc:341] T bf5ce45c1617423094e79ec99b51cdab P e70eeb6d81fd488b9c80b9c6de85eccc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 53a97834ac9e48e8b3c5ad90490c9d25 (127.18.45.195:45699): Not found: Tablet not found: bf5ce45c1617423094e79ec99b51cdab
W20260806 22:48:43.172163 9869 leader_election.cc:341] T bf5ce45c1617423094e79ec99b51cdab P e70eeb6d81fd488b9c80b9c6de85eccc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4756eb5819a543f4bbf440e26deb8284 (127.18.45.194:34433): Not found: Tablet not found: bf5ce45c1617423094e79ec99b51cdab
I20260806 22:48:43.172227 9869 leader_election.cc:302] T bf5ce45c1617423094e79ec99b51cdab P e70eeb6d81fd488b9c80b9c6de85eccc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e70eeb6d81fd488b9c80b9c6de85eccc; no voters: 4756eb5819a543f4bbf440e26deb8284, 53a97834ac9e48e8b3c5ad90490c9d25
I20260806 22:48:43.172390 10394 raft_consensus.cc:2726] T bf5ce45c1617423094e79ec99b51cdab P e70eeb6d81fd488b9c80b9c6de85eccc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 22:48:43.290967 10400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:43.290677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:32823 (local address 127.18.45.195:43797)
0806 22:48:43.290856 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:43.290913 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":102,"threads_started":1}
I20260806 22:48:44.065282 10401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:44.064955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:32823 (local address 127.18.45.193:46975)
0806 22:48:44.065156 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:44.065212 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":103,"threads_started":1}
I20260806 22:48:44.124007 10403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:44.123754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:32823 (local address 127.18.45.194:36027)
0806 22:48:44.123904 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:44.123947 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":80,"threads_started":1}
I20260806 22:48:44.291838 10404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:44.291545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:32823 (local address 127.18.45.195:37721)
0806 22:48:44.291714 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:44.291758 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":90,"threads_started":1}
I20260806 22:48:44.814553 10405 raft_consensus.cc:482] T bf5ce45c1617423094e79ec99b51cdab P e70eeb6d81fd488b9c80b9c6de85eccc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:48:44.814693 10405 raft_consensus.cc:504] T bf5ce45c1617423094e79ec99b51cdab P e70eeb6d81fd488b9c80b9c6de85eccc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4756eb5819a543f4bbf440e26deb8284" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 34433 } } peers { permanent_uuid: "53a97834ac9e48e8b3c5ad90490c9d25" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 45699 } } peers { permanent_uuid: "e70eeb6d81fd488b9c80b9c6de85eccc" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 41625 } }
I20260806 22:48:44.814998 10405 leader_election.cc:288] T bf5ce45c1617423094e79ec99b51cdab P e70eeb6d81fd488b9c80b9c6de85eccc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4756eb5819a543f4bbf440e26deb8284 (127.18.45.194:34433), 53a97834ac9e48e8b3c5ad90490c9d25 (127.18.45.195:45699)
W20260806 22:48:44.815481 9869 leader_election.cc:341] T bf5ce45c1617423094e79ec99b51cdab P e70eeb6d81fd488b9c80b9c6de85eccc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4756eb5819a543f4bbf440e26deb8284 (127.18.45.194:34433): Not found: Tablet not found: bf5ce45c1617423094e79ec99b51cdab
W20260806 22:48:44.815583 9868 leader_election.cc:341] T bf5ce45c1617423094e79ec99b51cdab P e70eeb6d81fd488b9c80b9c6de85eccc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 53a97834ac9e48e8b3c5ad90490c9d25 (127.18.45.195:45699): Not found: Tablet not found: bf5ce45c1617423094e79ec99b51cdab
I20260806 22:48:44.815701 9868 leader_election.cc:302] T bf5ce45c1617423094e79ec99b51cdab P e70eeb6d81fd488b9c80b9c6de85eccc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e70eeb6d81fd488b9c80b9c6de85eccc; no voters: 4756eb5819a543f4bbf440e26deb8284, 53a97834ac9e48e8b3c5ad90490c9d25
I20260806 22:48:44.815816 10405 raft_consensus.cc:2726] T bf5ce45c1617423094e79ec99b51cdab P e70eeb6d81fd488b9c80b9c6de85eccc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 22:48:45.066378 10406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:45.065986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:32823 (local address 127.18.45.193:39279)
0806 22:48:45.066196 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:45.066257 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":95,"threads_started":1}
I20260806 22:48:45.124956 10407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:45.124637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:32823 (local address 127.18.45.194:54729)
0806 22:48:45.124834 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:45.124880 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":89,"threads_started":1}
I20260806 22:48:45.292893 10408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:45.292493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:32823 (local address 127.18.45.195:52939)
0806 22:48:45.292730 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:45.292785 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":131,"threads_started":1}
I20260806 22:48:46.067432 10409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:46.067148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:32823 (local address 127.18.45.193:41053)
0806 22:48:46.067311 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:46.067359 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":85,"threads_started":1}
I20260806 22:48:46.125926 10410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:46.125650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:32823 (local address 127.18.45.194:43133)
0806 22:48:46.125817 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:46.125865 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":85,"threads_started":1}
I20260806 22:48:46.293741 10412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:46.293463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:32823 (local address 127.18.45.195:57739)
0806 22:48:46.293639 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:46.293682 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":110,"threads_started":1}
I20260806 22:48:47.068334 10413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:47.068052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:32823 (local address 127.18.45.193:47477)
0806 22:48:47.068214 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:47.068256 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":87,"threads_started":1}
I20260806 22:48:47.126870 10415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:47.126543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:32823 (local address 127.18.45.194:37357)
0806 22:48:47.126742 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:47.126798 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":91,"threads_started":1}
I20260806 22:48:47.294572 10416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:47.294214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:32823 (local address 127.18.45.195:54633)
0806 22:48:47.294414 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:47.294499 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":110,"threads_started":1}
I20260806 22:48:48.061406 10417 raft_consensus.cc:482] T bf5ce45c1617423094e79ec99b51cdab P e70eeb6d81fd488b9c80b9c6de85eccc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:48:48.061657 10417 raft_consensus.cc:504] T bf5ce45c1617423094e79ec99b51cdab P e70eeb6d81fd488b9c80b9c6de85eccc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4756eb5819a543f4bbf440e26deb8284" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 34433 } } peers { permanent_uuid: "53a97834ac9e48e8b3c5ad90490c9d25" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 45699 } } peers { permanent_uuid: "e70eeb6d81fd488b9c80b9c6de85eccc" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 41625 } }
I20260806 22:48:48.061933 10417 leader_election.cc:288] T bf5ce45c1617423094e79ec99b51cdab P e70eeb6d81fd488b9c80b9c6de85eccc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4756eb5819a543f4bbf440e26deb8284 (127.18.45.194:34433), 53a97834ac9e48e8b3c5ad90490c9d25 (127.18.45.195:45699)
W20260806 22:48:48.062491 9869 leader_election.cc:341] T bf5ce45c1617423094e79ec99b51cdab P e70eeb6d81fd488b9c80b9c6de85eccc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4756eb5819a543f4bbf440e26deb8284 (127.18.45.194:34433): Not found: Tablet not found: bf5ce45c1617423094e79ec99b51cdab
W20260806 22:48:48.062641 9868 leader_election.cc:341] T bf5ce45c1617423094e79ec99b51cdab P e70eeb6d81fd488b9c80b9c6de85eccc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 53a97834ac9e48e8b3c5ad90490c9d25 (127.18.45.195:45699): Not found: Tablet not found: bf5ce45c1617423094e79ec99b51cdab
I20260806 22:48:48.062683 9868 leader_election.cc:302] T bf5ce45c1617423094e79ec99b51cdab P e70eeb6d81fd488b9c80b9c6de85eccc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e70eeb6d81fd488b9c80b9c6de85eccc; no voters: 4756eb5819a543f4bbf440e26deb8284, 53a97834ac9e48e8b3c5ad90490c9d25
I20260806 22:48:48.062774 10417 raft_consensus.cc:2726] T bf5ce45c1617423094e79ec99b51cdab P e70eeb6d81fd488b9c80b9c6de85eccc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 22:48:48.069229 10418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:48.068919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:32823 (local address 127.18.45.193:51299)
0806 22:48:48.069121 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:48.069170 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":129,"threads_started":1}
I20260806 22:48:48.127944 10419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:48.127635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:32823 (local address 127.18.45.194:49131)
0806 22:48:48.127823 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:48.127876 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":111,"threads_started":1}
I20260806 22:48:48.295603 10420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:48.295303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:32823 (local address 127.18.45.195:47503)
0806 22:48:48.295470 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:48.295519 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":93,"threads_started":1}
I20260806 22:48:49.070283 10421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:49.069979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:32823 (local address 127.18.45.193:53985)
0806 22:48:49.070153 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:49.070208 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":95,"threads_started":1}
I20260806 22:48:49.128791 10422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:49.128492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:32823 (local address 127.18.45.194:38043)
0806 22:48:49.128688 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:49.128729 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":122,"threads_started":1}
I20260806 22:48:49.296480 10423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:49.296201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:32823 (local address 127.18.45.195:43819)
0806 22:48:49.296368 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:49.296410 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":85,"threads_started":1}
I20260806 22:48:50.071506 10425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:50.071149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:32823 (local address 127.18.45.193:41619)
0806 22:48:50.071371 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:50.071432 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":97,"threads_started":1}
I20260806 22:48:50.129827 10426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:50.129497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:32823 (local address 127.18.45.194:41583)
0806 22:48:50.129699 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:50.129758 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":91,"threads_started":1}
I20260806 22:48:50.297626 10428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:50.297282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:32823 (local address 127.18.45.195:39857)
0806 22:48:50.297490 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:50.297551 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":99,"threads_started":1}
I20260806 22:48:51.072631 10430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:51.072220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:32823 (local address 127.18.45.193:33359)
0806 22:48:51.072471 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:51.072532 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":131,"threads_started":1}
I20260806 22:48:51.130993 10431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:51.130631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:32823 (local address 127.18.45.194:56147)
0806 22:48:51.130863 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:51.130920 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":119,"threads_started":1}
I20260806 22:48:51.298625 10433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:51.298310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:32823 (local address 127.18.45.195:37689)
0806 22:48:51.298493 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:51.298549 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":87,"threads_started":1}
I20260806 22:48:51.679301 10434 raft_consensus.cc:482] T bf5ce45c1617423094e79ec99b51cdab P e70eeb6d81fd488b9c80b9c6de85eccc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:48:51.679454 10434 raft_consensus.cc:504] T bf5ce45c1617423094e79ec99b51cdab P e70eeb6d81fd488b9c80b9c6de85eccc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4756eb5819a543f4bbf440e26deb8284" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 34433 } } peers { permanent_uuid: "53a97834ac9e48e8b3c5ad90490c9d25" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 45699 } } peers { permanent_uuid: "e70eeb6d81fd488b9c80b9c6de85eccc" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 41625 } }
I20260806 22:48:51.679689 10434 leader_election.cc:288] T bf5ce45c1617423094e79ec99b51cdab P e70eeb6d81fd488b9c80b9c6de85eccc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4756eb5819a543f4bbf440e26deb8284 (127.18.45.194:34433), 53a97834ac9e48e8b3c5ad90490c9d25 (127.18.45.195:45699)
W20260806 22:48:51.680287 9868 leader_election.cc:341] T bf5ce45c1617423094e79ec99b51cdab P e70eeb6d81fd488b9c80b9c6de85eccc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 53a97834ac9e48e8b3c5ad90490c9d25 (127.18.45.195:45699): Not found: Tablet not found: bf5ce45c1617423094e79ec99b51cdab
W20260806 22:48:51.680393 9869 leader_election.cc:341] T bf5ce45c1617423094e79ec99b51cdab P e70eeb6d81fd488b9c80b9c6de85eccc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4756eb5819a543f4bbf440e26deb8284 (127.18.45.194:34433): Not found: Tablet not found: bf5ce45c1617423094e79ec99b51cdab
I20260806 22:48:51.680493 9869 leader_election.cc:302] T bf5ce45c1617423094e79ec99b51cdab P e70eeb6d81fd488b9c80b9c6de85eccc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e70eeb6d81fd488b9c80b9c6de85eccc; no voters: 4756eb5819a543f4bbf440e26deb8284, 53a97834ac9e48e8b3c5ad90490c9d25
I20260806 22:48:51.680599 10434 raft_consensus.cc:2726] T bf5ce45c1617423094e79ec99b51cdab P e70eeb6d81fd488b9c80b9c6de85eccc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 22:48:52.073730 10435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:52.073399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:32823 (local address 127.18.45.193:50187)
0806 22:48:52.073605 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:52.073653 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":101,"threads_started":1}
I20260806 22:48:52.132081 10436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:52.131674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:32823 (local address 127.18.45.194:42813)
0806 22:48:52.131887 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:52.131980 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":95,"threads_started":1}
I20260806 22:48:52.300637 10437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:52.299280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:32823 (local address 127.18.45.195:43675)
0806 22:48:52.300516 (+ 1236us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:52.300563 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1213,"thread_start_us":1140,"threads_started":1}
I20260806 22:48:53.074738 10438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:53.074388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:32823 (local address 127.18.45.193:36353)
0806 22:48:53.074600 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:53.074645 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":105,"threads_started":1}
I20260806 22:48:53.133162 10439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:53.132807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:32823 (local address 127.18.45.194:36603)
0806 22:48:53.133013 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:53.133089 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":94,"threads_started":1}
I20260806 22:48:53.301554 10440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:53.301168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:32823 (local address 127.18.45.195:36559)
0806 22:48:53.301400 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:53.301482 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":131,"threads_started":1}
I20260806 22:48:54.075749 10442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:54.075407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:32823 (local address 127.18.45.193:39471)
0806 22:48:54.075622 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:54.075673 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":131,"threads_started":1}
I20260806 22:48:54.134142 10444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:54.133825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:32823 (local address 127.18.45.194:41165)
0806 22:48:54.134010 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:54.134059 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":84,"threads_started":1}
I20260806 22:48:54.302402 10445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:54.302104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:32823 (local address 127.18.45.195:40465)
0806 22:48:54.302273 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:54.302335 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":85,"threads_started":1}
I20260806 22:48:54.390215 10449 raft_consensus.cc:482] T bf5ce45c1617423094e79ec99b51cdab P e70eeb6d81fd488b9c80b9c6de85eccc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:48:54.390353 10449 raft_consensus.cc:504] T bf5ce45c1617423094e79ec99b51cdab P e70eeb6d81fd488b9c80b9c6de85eccc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4756eb5819a543f4bbf440e26deb8284" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 34433 } } peers { permanent_uuid: "53a97834ac9e48e8b3c5ad90490c9d25" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 45699 } } peers { permanent_uuid: "e70eeb6d81fd488b9c80b9c6de85eccc" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 41625 } }
I20260806 22:48:54.390666 10449 leader_election.cc:288] T bf5ce45c1617423094e79ec99b51cdab P e70eeb6d81fd488b9c80b9c6de85eccc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4756eb5819a543f4bbf440e26deb8284 (127.18.45.194:34433), 53a97834ac9e48e8b3c5ad90490c9d25 (127.18.45.195:45699)
W20260806 22:48:54.391214 9868 leader_election.cc:341] T bf5ce45c1617423094e79ec99b51cdab P e70eeb6d81fd488b9c80b9c6de85eccc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 53a97834ac9e48e8b3c5ad90490c9d25 (127.18.45.195:45699): Not found: Tablet not found: bf5ce45c1617423094e79ec99b51cdab
W20260806 22:48:54.391378 9869 leader_election.cc:341] T bf5ce45c1617423094e79ec99b51cdab P e70eeb6d81fd488b9c80b9c6de85eccc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4756eb5819a543f4bbf440e26deb8284 (127.18.45.194:34433): Not found: Tablet not found: bf5ce45c1617423094e79ec99b51cdab
I20260806 22:48:54.391463 9869 leader_election.cc:302] T bf5ce45c1617423094e79ec99b51cdab P e70eeb6d81fd488b9c80b9c6de85eccc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e70eeb6d81fd488b9c80b9c6de85eccc; no voters: 4756eb5819a543f4bbf440e26deb8284, 53a97834ac9e48e8b3c5ad90490c9d25
I20260806 22:48:54.391592 10449 raft_consensus.cc:2726] T bf5ce45c1617423094e79ec99b51cdab P e70eeb6d81fd488b9c80b9c6de85eccc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 22:48:55.076710 10450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:55.076375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:32823 (local address 127.18.45.193:37541)
0806 22:48:55.076583 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:55.076637 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":98,"threads_started":1}
I20260806 22:48:55.135362 10451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:55.134930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:32823 (local address 127.18.45.194:45761)
0806 22:48:55.135223 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:55.135287 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":176,"threads_started":1}
I20260806 22:48:55.303413 10452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:55.303085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:32823 (local address 127.18.45.195:41735)
0806 22:48:55.303286 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:55.303343 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":87,"threads_started":1}
I20260806 22:48:56.077759 10454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:56.077391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:32823 (local address 127.18.45.193:51937)
0806 22:48:56.077615 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:56.077678 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":95,"threads_started":1}
I20260806 22:48:56.136449 10455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:56.136117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:32823 (local address 127.18.45.194:36995)
0806 22:48:56.136311 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:56.136371 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":91,"threads_started":1}
I20260806 22:48:56.304455 10456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:56.304101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:32823 (local address 127.18.45.195:59573)
0806 22:48:56.304310 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:56.304379 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":111,"threads_started":1}
I20260806 22:48:57.079068 10459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:57.078623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:32823 (local address 127.18.45.193:50585)
0806 22:48:57.078868 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:57.078924 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":116,"threads_started":1}
I20260806 22:48:57.137431 10461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:57.137139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:32823 (local address 127.18.45.194:56467)
0806 22:48:57.137318 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:57.137363 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":112,"threads_started":1}
I20260806 22:48:57.294691 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 9795
I20260806 22:48:57.306115 10462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:57.305334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:32823 (local address 127.18.45.195:42335)
0806 22:48:57.305984 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:57.306044 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.254:32823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":622,"thread_start_us":413,"threads_started":1}
I20260806 22:48:57.313354 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 10024
I20260806 22:48:57.325590 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 10203
2026-08-06T22:48:57Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59356 ms)
[----------] 30 tests from SecurityITest (261527 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:48:57 dist-test-slave-h3ft krb5kdc[10466](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:48:57 dist-test-slave-h3ft krb5kdc[10466](info): set up 2 sockets
Aug 06 22:48:57 dist-test-slave-h3ft krb5kdc[10466](info): commencing operation
krb5kdc: starting...
W20260806 22:48:58.615237 18615 mini_kdc.cc:121] Time spent starting KDC: real 1.257s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Aug 06 22:48:58 dist-test-slave-h3ft krb5kdc[10466](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056538, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:48:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:48:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.45.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab.
Entry for principal kudu/127.18.45.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab.
I20260806 22:48:58.766568 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:36079
--webserver_interface=127.18.45.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:39439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.45.254:36079
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:48:58.864640 10488 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:48:58.864943 10488 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:48:58.864988 10488 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:48:58.866220 10488 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:48:58.866292 10488 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:48:58.866328 10488 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:48:58.866361 10488 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:48:58.866386 10488 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:48:58.868100 10488 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:39439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.45.254:36079
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:36079
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.45.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10488
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:48:58.868425 10488 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:48:58.868698 10488 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:48:58.874830 10496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:48:58.874868 10499 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:48:58.874882 10495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:48:58.875041 10488 server_base.cc:1034] running on GCE node
I20260806 22:48:58.875530 10488 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:48:58.875885 10488 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:48:58.877079 10488 hybrid_clock.cc:648] HybridClock initialized: now 1786056538877056 us; error 45 us; skew 500 ppm
Aug 06 22:48:58 dist-test-slave-h3ft krb5kdc[10466](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056538, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:48:58.880141 10488 init.cc:377] Logged in from keytab as kudu/127.18.45.254@KRBTEST.COM (short username kudu)
I20260806 22:48:58.880712 10488 webserver.cc:460] Webserver started at http://127.18.45.254:44953/ using document root <none> and password file <none>
I20260806 22:48:58.880982 10488 fs_manager.cc:359] Metadata directory not provided
I20260806 22:48:58.881055 10488 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:48:58.881218 10488 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:48:58.882477 10488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/minicluster-data/master-0/data/instance:
uuid: "c5aad1fbc4be4a338b4e868af0b7ff66"
format_stamp: "Formatted at 2026-08-06 22:48:58 on dist-test-slave-h3ft"
server_key: "783ee40f4a1f7c94d1d57cb2e7c7543a"
server_key_iv: "8833d05cdafe52e0288e6d29b6250315"
server_key_version: "encryptionkey@0"
I20260806 22:48:58.882905 10488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/minicluster-data/master-0/wal/instance:
uuid: "c5aad1fbc4be4a338b4e868af0b7ff66"
format_stamp: "Formatted at 2026-08-06 22:48:58 on dist-test-slave-h3ft"
server_key: "783ee40f4a1f7c94d1d57cb2e7c7543a"
server_key_iv: "8833d05cdafe52e0288e6d29b6250315"
server_key_version: "encryptionkey@0"
I20260806 22:48:58.884545 10488 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:48:58.885541 10508 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:48:58.885864 10488 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:48:58.885999 10488 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/minicluster-data/master-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/minicluster-data/master-0/wal
uuid: "c5aad1fbc4be4a338b4e868af0b7ff66"
format_stamp: "Formatted at 2026-08-06 22:48:58 on dist-test-slave-h3ft"
server_key: "783ee40f4a1f7c94d1d57cb2e7c7543a"
server_key_iv: "8833d05cdafe52e0288e6d29b6250315"
server_key_version: "encryptionkey@0"
I20260806 22:48:58.886091 10488 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:48:58.906702 10488 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:48:58.909307 10488 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:48:58.909519 10488 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:48:58.914711 10488 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.254:36079
I20260806 22:48:58.914748 10575 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.254:36079 every 8 connection(s)
I20260806 22:48:58.915207 10488 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
I20260806 22:48:58.916318 10576 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:48:58.919369 10576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5aad1fbc4be4a338b4e868af0b7ff66: Bootstrap starting.
I20260806 22:48:58.920091 10576 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c5aad1fbc4be4a338b4e868af0b7ff66: Neither blocks nor log segments found. Creating new log.
I20260806 22:48:58.920408 10576 log.cc:824] T 00000000000000000000000000000000 P c5aad1fbc4be4a338b4e868af0b7ff66: Log is configured to *not* fsync() on all Append() calls
I20260806 22:48:58.921247 10576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5aad1fbc4be4a338b4e868af0b7ff66: No bootstrap required, opened a new log
I20260806 22:48:58.923100 10576 raft_consensus.cc:348] T 00000000000000000000000000000000 P c5aad1fbc4be4a338b4e868af0b7ff66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5aad1fbc4be4a338b4e868af0b7ff66" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 36079 } }
I20260806 22:48:58.923274 10576 raft_consensus.cc:374] T 00000000000000000000000000000000 P c5aad1fbc4be4a338b4e868af0b7ff66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:48:58.923300 10576 raft_consensus.cc:729] T 00000000000000000000000000000000 P c5aad1fbc4be4a338b4e868af0b7ff66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5aad1fbc4be4a338b4e868af0b7ff66, State: Initialized, Role: FOLLOWER
I20260806 22:48:58.923429 10576 consensus_queue.cc:260] T 00000000000000000000000000000000 P c5aad1fbc4be4a338b4e868af0b7ff66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5aad1fbc4be4a338b4e868af0b7ff66" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 36079 } }
I20260806 22:48:58.923491 10576 raft_consensus.cc:388] T 00000000000000000000000000000000 P c5aad1fbc4be4a338b4e868af0b7ff66 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:48:58.923514 10576 raft_consensus.cc:482] T 00000000000000000000000000000000 P c5aad1fbc4be4a338b4e868af0b7ff66 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:48:58.923583 10576 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c5aad1fbc4be4a338b4e868af0b7ff66 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:48:58.923794 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 10488
I20260806 22:48:58.923928 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/minicluster-data/master-0/wal/instance
I20260806 22:48:58.924168 18615 external_mini_cluster.cc:1428] Setting key 5214ce25603556befbff5698cded7e10
I20260806 22:48:58.924446 10576 raft_consensus.cc:504] T 00000000000000000000000000000000 P c5aad1fbc4be4a338b4e868af0b7ff66 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5aad1fbc4be4a338b4e868af0b7ff66" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 36079 } }
I20260806 22:48:58.924585 10576 leader_election.cc:302] T 00000000000000000000000000000000 P c5aad1fbc4be4a338b4e868af0b7ff66 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c5aad1fbc4be4a338b4e868af0b7ff66; no voters:
I20260806 22:48:58.924808 10576 leader_election.cc:288] T 00000000000000000000000000000000 P c5aad1fbc4be4a338b4e868af0b7ff66 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:48:58.924870 10580 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c5aad1fbc4be4a338b4e868af0b7ff66 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:48:58.925119 10580 raft_consensus.cc:686] T 00000000000000000000000000000000 P c5aad1fbc4be4a338b4e868af0b7ff66 [term 1 LEADER]: Becoming Leader. State: Replica: c5aad1fbc4be4a338b4e868af0b7ff66, State: Running, Role: LEADER
I20260806 22:48:58.925243 10576 sys_catalog.cc:564] T 00000000000000000000000000000000 P c5aad1fbc4be4a338b4e868af0b7ff66 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:48:58.925294 10580 consensus_queue.cc:237] T 00000000000000000000000000000000 P c5aad1fbc4be4a338b4e868af0b7ff66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5aad1fbc4be4a338b4e868af0b7ff66" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 36079 } }
I20260806 22:48:58.926004 10583 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5aad1fbc4be4a338b4e868af0b7ff66 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c5aad1fbc4be4a338b4e868af0b7ff66" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5aad1fbc4be4a338b4e868af0b7ff66" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 36079 } } }
I20260806 22:48:58.926102 10583 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5aad1fbc4be4a338b4e868af0b7ff66 [sys.catalog]: This master's current role is: LEADER
I20260806 22:48:58.926376 10586 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5aad1fbc4be4a338b4e868af0b7ff66 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c5aad1fbc4be4a338b4e868af0b7ff66. Latest consensus state: current_term: 1 leader_uuid: "c5aad1fbc4be4a338b4e868af0b7ff66" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5aad1fbc4be4a338b4e868af0b7ff66" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 36079 } } }
I20260806 22:48:58.926550 10586 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5aad1fbc4be4a338b4e868af0b7ff66 [sys.catalog]: This master's current role is: LEADER
I20260806 22:48:58.926756 10595 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:48:58.927467 10595 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:48:58.929369 10595 catalog_manager.cc:1347] Generated new cluster ID: 9621bce2680a40d4a4e8645d6076fa3f
I20260806 22:48:58.929455 10595 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:48:58 dist-test-slave-h3ft krb5kdc[10466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056538, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:48:58.937410 10582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:58.924769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38100 (local address 127.18.45.254:36079)
0806 22:48:58.924991 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0806 22:48:58.925005 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 22:48:58.925056 (+ 51us) server_negotiation.cc:407] Connection header received
0806 22:48:58.925285 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:48:58.925294 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:48:58.925365 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:48:58.925526 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:48:58.926646 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:58.928266 (+ 1620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:58.929378 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:58.929929 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:58.929996 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:58.932858 (+ 2862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:48:58.932863 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:48:58.932863 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:48:58.932884 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:48:58.935544 (+ 2660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:48:58.935968 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:48:58.935974 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:48:58.935974 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:48:58.936108 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:48:58.936351 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:48:58.936352 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:48:58.936353 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:48:58.936660 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:48:58.936868 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0806 22:48:58.936946 (+ 78us) server_negotiation.cc:299] Negotiation successful
0806 22:48:58.937110 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":76,"threads_started":1}
I20260806 22:48:58.944595 10595 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:48:58.945374 10595 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:48:58.963564 10595 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c5aad1fbc4be4a338b4e868af0b7ff66: Generated new TSK 0
I20260806 22:48:58.963946 10595 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.45.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab.
Entry for principal kudu/127.18.45.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab.
I20260806 22:48:59.002748 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.193:0
--local_ip_for_outbound_sockets=127.18.45.193
--webserver_interface=127.18.45.193
--webserver_port=0
--tserver_master_addrs=127.18.45.254:36079
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab
--principal=kudu/127.18.45.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:39439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:48:59.107111 10610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:48:59.107424 10610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:48:59.107470 10610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:48:59.108774 10610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:48:59.108861 10610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:48:59.108915 10610 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.193
I20260806 22:48:59.110654 10610 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:39439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab
--principal=kudu/127.18.45.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.45.193
--webserver_port=0
--tserver_master_addrs=127.18.45.254:36079
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10610
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.193
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:48:59.111037 10610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:48:59.111351 10610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:48:59.117395 10617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:48:59.117422 10618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:48:59.117429 10621 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:48:59.117688 10610 server_base.cc:1034] running on GCE node
I20260806 22:48:59.117952 10610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:48:59.118194 10610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:48:59.119369 10610 hybrid_clock.cc:648] HybridClock initialized: now 1786056539119350 us; error 49 us; skew 500 ppm
Aug 06 22:48:59 dist-test-slave-h3ft krb5kdc[10466](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056539, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:48:59.122272 10610 init.cc:377] Logged in from keytab as kudu/127.18.45.193@KRBTEST.COM (short username kudu)
I20260806 22:48:59.122769 10610 webserver.cc:460] Webserver started at http://127.18.45.193:37061/ using document root <none> and password file <none>
I20260806 22:48:59.123087 10610 fs_manager.cc:359] Metadata directory not provided
I20260806 22:48:59.123167 10610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:48:59.123337 10610 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:48:59.124594 10610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/minicluster-data/ts-0/data/instance:
uuid: "fc97311847394bab985454d6f623a66b"
format_stamp: "Formatted at 2026-08-06 22:48:59 on dist-test-slave-h3ft"
server_key: "9cb7c3ffea65bfb1161aba5d37f18a52"
server_key_iv: "5c8cf7b4b22e0e6ca2cc5a5ff7ad676b"
server_key_version: "encryptionkey@0"
I20260806 22:48:59.125011 10610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/minicluster-data/ts-0/wal/instance:
uuid: "fc97311847394bab985454d6f623a66b"
format_stamp: "Formatted at 2026-08-06 22:48:59 on dist-test-slave-h3ft"
server_key: "9cb7c3ffea65bfb1161aba5d37f18a52"
server_key_iv: "5c8cf7b4b22e0e6ca2cc5a5ff7ad676b"
server_key_version: "encryptionkey@0"
I20260806 22:48:59.126770 10610 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:48:59.127720 10628 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:48:59.127926 10610 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:48:59.128041 10610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/minicluster-data/ts-0/wal
uuid: "fc97311847394bab985454d6f623a66b"
format_stamp: "Formatted at 2026-08-06 22:48:59 on dist-test-slave-h3ft"
server_key: "9cb7c3ffea65bfb1161aba5d37f18a52"
server_key_iv: "5c8cf7b4b22e0e6ca2cc5a5ff7ad676b"
server_key_version: "encryptionkey@0"
I20260806 22:48:59.128134 10610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:48:59.137743 10610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:48:59.140005 10610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:48:59.140192 10610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:48:59.140470 10610 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:48:59.140825 10610 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:48:59.140882 10610 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:48:59.140919 10610 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:48:59.140967 10610 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:48:59.148386 10610 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.193:36433
I20260806 22:48:59.148484 10795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.193:36433 every 8 connection(s)
I20260806 22:48:59.148864 10610 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
I20260806 22:48:59.150524 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 10610
I20260806 22:48:59.150674 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/minicluster-data/ts-0/wal/instance
I20260806 22:48:59.150940 18615 external_mini_cluster.cc:1428] Setting key b69de9d5c04f959b3c3090771ddba078
Aug 06 22:48:59 dist-test-slave-h3ft krb5kdc[10466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056539, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.193@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:48:59.160810 10582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:59.149782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:44501 (local address 127.18.45.254:36079)
0806 22:48:59.149873 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0806 22:48:59.149876 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:48:59.150177 (+ 301us) server_negotiation.cc:407] Connection header received
0806 22:48:59.150769 (+ 592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:48:59.150770 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:48:59.150792 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:48:59.150852 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:48:59.152231 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:59.152806 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:59.153998 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:59.154310 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:59.154379 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:59.157336 (+ 2957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:48:59.157338 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:48:59.157338 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:48:59.157346 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:48:59.158991 (+ 1645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:48:59.159598 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:48:59.159599 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:48:59.159600 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:48:59.159658 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:48:59.160044 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:48:59.160045 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:48:59.160045 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:48:59.160177 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:48:59.160261 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 22:48:59.160638 (+ 377us) server_negotiation.cc:299] Negotiation successful
0806 22:48:59.160731 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 22:48:59.161105 10800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:59.149761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:36079 (local address 127.18.45.193:44501)
0806 22:48:59.149955 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:59.150006 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 22:48:59.150639 (+ 633us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:48:59.150939 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:48:59.150944 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:48:59.151845 (+ 901us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:48:59.152126 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:48:59.152134 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:48:59.152914 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:48:59.152915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:48:59.153801 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:48:59.153803 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:48:59.154407 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:48:59.154409 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:48:59.154586 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:59.155261 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:48:59.155273 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:48:59.157225 (+ 1952us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:48:59.159141 (+ 1916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:48:59.159150 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:48:59.159153 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:48:59.159518 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:48:59.159730 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:48:59.159731 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:48:59.159731 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:48:59.159926 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:48:59.160260 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:48:59.160269 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:48:59.160473 (+ 204us) client_negotiation.cc:769] Sending connection context
0806 22:48:59.160605 (+ 132us) client_negotiation.cc:240] Negotiation successful
0806 22:48:59.160807 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":102,"threads_started":1}
I20260806 22:48:59.161763 10796 heartbeater.cc:347] Connected to a master server at 127.18.45.254:36079
I20260806 22:48:59.161890 10796 heartbeater.cc:464] Registering TS with master...
I20260806 22:48:59.162144 10796 heartbeater.cc:511] Master 127.18.45.254:36079 requested a full tablet report, sending...
I20260806 22:48:59.162809 10532 ts_manager.cc:194] Registered new tserver with Master: fc97311847394bab985454d6f623a66b (127.18.45.193:36433)
WARNING: no policy specified for kudu/127.18.45.194@KRBTEST.COM; defaulting to no policy
I20260806 22:48:59.163543 10532 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.45.193@KRBTEST.COM'} at 127.18.45.193:44501
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab.
Entry for principal kudu/127.18.45.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab.
I20260806 22:48:59.183171 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.194:0
--local_ip_for_outbound_sockets=127.18.45.194
--webserver_interface=127.18.45.194
--webserver_port=0
--tserver_master_addrs=127.18.45.254:36079
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab
--principal=kudu/127.18.45.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:39439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:48:59.278640 10803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:48:59.278916 10803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:48:59.278956 10803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:48:59.280194 10803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:48:59.280270 10803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:48:59.280314 10803 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.194
I20260806 22:48:59.281872 10803 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:39439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab
--principal=kudu/127.18.45.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.45.194
--webserver_port=0
--tserver_master_addrs=127.18.45.254:36079
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10803
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.194
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:48:59.282163 10803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:48:59.282429 10803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:48:59.288278 10814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:48:59.288285 10809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:48:59.288270 10816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:48:59.288594 10803 server_base.cc:1034] running on GCE node
I20260806 22:48:59.288869 10803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:48:59.289192 10803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:48:59.290352 10803 hybrid_clock.cc:648] HybridClock initialized: now 1786056539290325 us; error 39 us; skew 500 ppm
Aug 06 22:48:59 dist-test-slave-h3ft krb5kdc[10466](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056539, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:48:59.293351 10803 init.cc:377] Logged in from keytab as kudu/127.18.45.194@KRBTEST.COM (short username kudu)
I20260806 22:48:59.293848 10803 webserver.cc:460] Webserver started at http://127.18.45.194:34217/ using document root <none> and password file <none>
I20260806 22:48:59.294135 10803 fs_manager.cc:359] Metadata directory not provided
I20260806 22:48:59.294211 10803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:48:59.294373 10803 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:48:59.295660 10803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/minicluster-data/ts-1/data/instance:
uuid: "ebe47a29283a42d7b3bca27f9b064ccb"
format_stamp: "Formatted at 2026-08-06 22:48:59 on dist-test-slave-h3ft"
server_key: "a5ff879ff9464986e2bf0821ecb48493"
server_key_iv: "7a9420639bdbf97aa093bad80264d5e2"
server_key_version: "encryptionkey@0"
I20260806 22:48:59.296078 10803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/minicluster-data/ts-1/wal/instance:
uuid: "ebe47a29283a42d7b3bca27f9b064ccb"
format_stamp: "Formatted at 2026-08-06 22:48:59 on dist-test-slave-h3ft"
server_key: "a5ff879ff9464986e2bf0821ecb48493"
server_key_iv: "7a9420639bdbf97aa093bad80264d5e2"
server_key_version: "encryptionkey@0"
I20260806 22:48:59.297562 10803 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 22:48:59.298449 10828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:48:59.298640 10803 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:48:59.298758 10803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/minicluster-data/ts-1/wal
uuid: "ebe47a29283a42d7b3bca27f9b064ccb"
format_stamp: "Formatted at 2026-08-06 22:48:59 on dist-test-slave-h3ft"
server_key: "a5ff879ff9464986e2bf0821ecb48493"
server_key_iv: "7a9420639bdbf97aa093bad80264d5e2"
server_key_version: "encryptionkey@0"
I20260806 22:48:59.298841 10803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:48:59.328253 10803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:48:59.330744 10803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:48:59.330935 10803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:48:59.331347 10803 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:48:59.331719 10803 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:48:59.331755 10803 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:48:59.331784 10803 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:48:59.331797 10803 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:48:59.338866 10803 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.194:36461
I20260806 22:48:59.338996 10987 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.194:36461 every 8 connection(s)
I20260806 22:48:59.339377 10803 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
I20260806 22:48:59.340480 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 10803
I20260806 22:48:59.340622 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/minicluster-data/ts-1/wal/instance
I20260806 22:48:59.340886 18615 external_mini_cluster.cc:1428] Setting key 8fd5adb5d36c63acc895220bc69eaeb9
Aug 06 22:48:59 dist-test-slave-h3ft krb5kdc[10466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056539, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.194@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:48:59.351199 10582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:59.340309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:38495 (local address 127.18.45.254:36079)
0806 22:48:59.340395 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0806 22:48:59.340398 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:48:59.340841 (+ 443us) server_negotiation.cc:407] Connection header received
0806 22:48:59.341606 (+ 765us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:48:59.341608 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:48:59.341622 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:48:59.341671 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:48:59.342822 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:59.343458 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:59.345023 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:59.345247 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:59.345313 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:59.347940 (+ 2627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:48:59.347941 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:48:59.347942 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:48:59.347951 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:48:59.349549 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:48:59.350032 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:48:59.350034 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:48:59.350034 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:48:59.350105 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:48:59.350383 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:48:59.350384 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:48:59.350384 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:48:59.350527 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:48:59.350613 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 22:48:59.350925 (+ 312us) server_negotiation.cc:299] Negotiation successful
0806 22:48:59.351087 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 22:48:59.351397 10990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:59.340290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:36079 (local address 127.18.45.194:38495)
0806 22:48:59.340535 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:59.340708 (+ 173us) client_negotiation.cc:174] Beginning negotiation
0806 22:48:59.341485 (+ 777us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:48:59.341754 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:48:59.341759 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:48:59.342442 (+ 683us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:48:59.342695 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:48:59.342701 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:48:59.343573 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:48:59.343574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:48:59.344905 (+ 1331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:48:59.344909 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:48:59.345342 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:48:59.345343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:48:59.345518 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:59.346164 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:48:59.346174 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:48:59.347823 (+ 1649us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:48:59.349657 (+ 1834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:48:59.349666 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:48:59.349669 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:48:59.349925 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:48:59.350185 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:48:59.350185 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:48:59.350186 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:48:59.350312 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:48:59.350605 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:48:59.350614 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:48:59.350779 (+ 165us) client_negotiation.cc:769] Sending connection context
0806 22:48:59.350892 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 22:48:59.351113 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":128,"threads_started":1}
WARNING: no policy specified for kudu/127.18.45.195@KRBTEST.COM; defaulting to no policy
I20260806 22:48:59.352035 10988 heartbeater.cc:347] Connected to a master server at 127.18.45.254:36079
I20260806 22:48:59.352169 10988 heartbeater.cc:464] Registering TS with master...
I20260806 22:48:59.352385 10988 heartbeater.cc:511] Master 127.18.45.254:36079 requested a full tablet report, sending...
I20260806 22:48:59.352909 10532 ts_manager.cc:194] Registered new tserver with Master: ebe47a29283a42d7b3bca27f9b064ccb (127.18.45.194:36461)
I20260806 22:48:59.353308 10532 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.45.194@KRBTEST.COM'} at 127.18.45.194:38495
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab.
Entry for principal kudu/127.18.45.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab.
I20260806 22:48:59.373932 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.195:0
--local_ip_for_outbound_sockets=127.18.45.195
--webserver_interface=127.18.45.195
--webserver_port=0
--tserver_master_addrs=127.18.45.254:36079
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab
--principal=kudu/127.18.45.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:39439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:48:59.469587 10993 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:48:59.469842 10993 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:48:59.469870 10993 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:48:59.471215 10993 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:48:59.471259 10993 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:48:59.471287 10993 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.195
I20260806 22:48:59.472929 10993 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:39439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab
--principal=kudu/127.18.45.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.45.195
--webserver_port=0
--tserver_master_addrs=127.18.45.254:36079
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10993
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.195
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:48:59.473238 10993 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:48:59.473507 10993 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:48:59.479672 11002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:48:59.479758 10993 server_base.cc:1034] running on GCE node
W20260806 22:48:59.479702 11001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:48:59.479692 11004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:48:59.480216 10993 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:48:59.480522 10993 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:48:59.481719 10993 hybrid_clock.cc:648] HybridClock initialized: now 1786056539481693 us; error 59 us; skew 500 ppm
Aug 06 22:48:59 dist-test-slave-h3ft krb5kdc[10466](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056539, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:48:59.485072 10993 init.cc:377] Logged in from keytab as kudu/127.18.45.195@KRBTEST.COM (short username kudu)
I20260806 22:48:59.485605 10993 webserver.cc:460] Webserver started at http://127.18.45.195:34737/ using document root <none> and password file <none>
I20260806 22:48:59.485889 10993 fs_manager.cc:359] Metadata directory not provided
I20260806 22:48:59.485965 10993 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:48:59.486132 10993 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:48:59.487435 10993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/minicluster-data/ts-2/data/instance:
uuid: "2a46ef543d324436a106039cda17bb2a"
format_stamp: "Formatted at 2026-08-06 22:48:59 on dist-test-slave-h3ft"
server_key: "c7594b0e55691a629bf437a1720846c8"
server_key_iv: "05307c775609268c0d94106eee846143"
server_key_version: "encryptionkey@0"
I20260806 22:48:59.487862 10993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/minicluster-data/ts-2/wal/instance:
uuid: "2a46ef543d324436a106039cda17bb2a"
format_stamp: "Formatted at 2026-08-06 22:48:59 on dist-test-slave-h3ft"
server_key: "c7594b0e55691a629bf437a1720846c8"
server_key_iv: "05307c775609268c0d94106eee846143"
server_key_version: "encryptionkey@0"
I20260806 22:48:59.489414 10993 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:48:59.490340 11016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:48:59.490552 10993 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:48:59.490677 10993 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/minicluster-data/ts-2/wal
uuid: "2a46ef543d324436a106039cda17bb2a"
format_stamp: "Formatted at 2026-08-06 22:48:59 on dist-test-slave-h3ft"
server_key: "c7594b0e55691a629bf437a1720846c8"
server_key_iv: "05307c775609268c0d94106eee846143"
server_key_version: "encryptionkey@0"
I20260806 22:48:59.490760 10993 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:48:59.508620 10993 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:48:59.511616 10993 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:48:59.511855 10993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:48:59.512223 10993 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:48:59.512732 10993 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:48:59.512787 10993 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:48:59.512825 10993 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:48:59.512852 10993 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:48:59.520977 10993 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.195:33789
I20260806 22:48:59.521076 11179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.195:33789 every 8 connection(s)
I20260806 22:48:59.521397 10993 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
Aug 06 22:48:59 dist-test-slave-h3ft krb5kdc[10466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056539, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.195@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:48:59.531039 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 10993
I20260806 22:48:59.531183 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/minicluster-data/ts-2/wal/instance
I20260806 22:48:59.531394 18615 external_mini_cluster.cc:1428] Setting key ed7361247f433048b1de1d8b58226ce2
I20260806 22:48:59.532085 10582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:59.522200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.195:60763 (local address 127.18.45.254:36079)
0806 22:48:59.522278 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0806 22:48:59.522280 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:48:59.522478 (+ 198us) server_negotiation.cc:407] Connection header received
0806 22:48:59.523144 (+ 666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:48:59.523145 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:48:59.523157 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:48:59.523223 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:48:59.524295 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:59.524845 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:59.526013 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:59.526346 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:59.526416 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:59.528839 (+ 2423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:48:59.528842 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:48:59.528842 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:48:59.528858 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:48:59.530289 (+ 1431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:48:59.530751 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:48:59.530752 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:48:59.530753 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:48:59.530848 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:48:59.531218 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:48:59.531220 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:48:59.531220 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:48:59.531376 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:48:59.531462 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 22:48:59.531876 (+ 414us) server_negotiation.cc:299] Negotiation successful
0806 22:48:59.531974 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 22:48:59.532348 11186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:59.522136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:36079 (local address 127.18.45.195:60763)
0806 22:48:59.522333 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:59.522381 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 22:48:59.522991 (+ 610us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:48:59.523310 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:48:59.523315 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:48:59.523951 (+ 636us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:48:59.524212 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:48:59.524217 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:48:59.524932 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:48:59.524933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:48:59.525922 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:48:59.525924 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:48:59.526442 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:48:59.526443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:48:59.526575 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:59.527169 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:48:59.527179 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:48:59.528724 (+ 1545us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:48:59.530378 (+ 1654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:48:59.530386 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:48:59.530389 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:48:59.530639 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:48:59.530926 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:48:59.530927 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:48:59.530927 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:48:59.531140 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:48:59.531450 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:48:59.531458 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:48:59.531648 (+ 190us) client_negotiation.cc:769] Sending connection context
0806 22:48:59.531779 (+ 131us) client_negotiation.cc:240] Negotiation successful
0806 22:48:59.532053 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":107,"threads_started":1}
I20260806 22:48:59.532920 11181 heartbeater.cc:347] Connected to a master server at 127.18.45.254:36079
I20260806 22:48:59.533046 11181 heartbeater.cc:464] Registering TS with master...
I20260806 22:48:59.533252 11181 heartbeater.cc:511] Master 127.18.45.254:36079 requested a full tablet report, sending...
I20260806 22:48:59.533715 10532 ts_manager.cc:194] Registered new tserver with Master: 2a46ef543d324436a106039cda17bb2a (127.18.45.195:33789)
I20260806 22:48:59.534147 10532 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.45.195@KRBTEST.COM'} at 127.18.45.195:60763
I20260806 22:48:59.543462 18615 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:48:59.552029 10582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:59.545132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38108 (local address 127.18.45.254:36079)
0806 22:48:59.545231 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0806 22:48:59.545233 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:48:59.545291 (+ 58us) server_negotiation.cc:407] Connection header received
0806 22:48:59.545346 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:48:59.545347 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:48:59.545357 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:48:59.545406 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:48:59.546055 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:59.546640 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:59.547450 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:59.547652 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:59.547713 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:59.549162 (+ 1449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:48:59.549163 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:48:59.549164 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:48:59.549170 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:48:59.550722 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:48:59.551210 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:48:59.551211 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:48:59.551212 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:48:59.551271 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:48:59.551489 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:48:59.551489 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:48:59.551490 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:48:59.551657 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:48:59.551745 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 22:48:59.551838 (+ 93us) server_negotiation.cc:299] Negotiation successful
0806 22:48:59.551906 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 22:48:59.553917 10532 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38108:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:48:59.556087 10532 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:48:59.564258 11198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:59.559185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.195:33789 (local address 127.0.0.1:39252)
0806 22:48:59.560109 (+ 924us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:59.560116 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0806 22:48:59.560159 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:48:59.560356 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:48:59.560361 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:48:59.560364 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:48:59.560480 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:48:59.560483 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:48:59.561606 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:48:59.561608 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:48:59.563032 (+ 1424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:48:59.563036 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:48:59.563936 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:48:59.563938 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:48:59.564005 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:59.564009 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:48:59.564061 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 22:48:59.564146 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":906,"mutex_wait_us":69,"thread_start_us":94,"threads_started":1}
I20260806 22:48:59.564271 11200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:59.559316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34550 (local address 127.18.45.193:36433)
0806 22:48:59.559752 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0806 22:48:59.559759 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:48:59.559989 (+ 230us) server_negotiation.cc:407] Connection header received
0806 22:48:59.560092 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:48:59.560094 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:48:59.560111 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:48:59.560222 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:48:59.560559 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:59.561730 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:59.562867 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:59.563667 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:59.563748 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:59.563819 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 22:48:59.564125 (+ 306us) server_negotiation.cc:299] Negotiation successful
0806 22:48:59.564170 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":120,"threads_started":1}
I20260806 22:48:59.564469 11201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:59.559466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.194:36461 (local address 127.0.0.1:41608)
0806 22:48:59.560210 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:59.560216 (+ 6us) client_negotiation.cc:174] Beginning negotiation
0806 22:48:59.560279 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:48:59.560584 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:48:59.560585 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:48:59.560587 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:48:59.560640 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:48:59.560641 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:48:59.562224 (+ 1583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:48:59.562225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:48:59.563285 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:48:59.563286 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:48:59.564219 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:48:59.564220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:48:59.564282 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:59.564287 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:48:59.564348 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 22:48:59.564398 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":733,"thread_start_us":94,"threads_started":1}
I20260806 22:48:59.564683 11204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:59.559628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41608 (local address 127.18.45.194:36461)
0806 22:48:59.559846 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0806 22:48:59.559851 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:48:59.560287 (+ 436us) server_negotiation.cc:407] Connection header received
0806 22:48:59.560358 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:48:59.560361 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:48:59.560379 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:48:59.560469 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:48:59.560731 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:59.562091 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:59.563371 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:59.564053 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:59.564121 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:59.564184 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 22:48:59.564531 (+ 347us) server_negotiation.cc:299] Negotiation successful
0806 22:48:59.564592 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":115,"threads_started":1}
I20260806 22:48:59.564898 11197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:59.559183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.193:36433 (local address 127.0.0.1:34550)
0806 22:48:59.559855 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:59.559906 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 22:48:59.559999 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:48:59.560355 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:48:59.560359 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:48:59.560363 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:48:59.560473 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:48:59.560475 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:48:59.561859 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:48:59.561860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:48:59.562770 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:48:59.562772 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:48:59.563759 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:48:59.563760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:48:59.563839 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:59.563854 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 22:48:59.564064 (+ 210us) client_negotiation.cc:240] Negotiation successful
0806 22:48:59.564828 (+ 764us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":640,"mutex_wait_us":703,"thread_start_us":101,"threads_started":1}
I20260806 22:48:59.565198 11202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:59.559415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39252 (local address 127.18.45.195:33789)
0806 22:48:59.559868 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0806 22:48:59.559872 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:48:59.560199 (+ 327us) server_negotiation.cc:407] Connection header received
0806 22:48:59.560256 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:48:59.560258 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:48:59.560275 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:48:59.560363 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:48:59.560579 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:59.561494 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:59.563116 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:59.563839 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:59.563926 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:59.563986 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 22:48:59.565031 (+ 1045us) server_negotiation.cc:299] Negotiation successful
0806 22:48:59.565100 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":147,"threads_started":1}
I20260806 22:48:59.565668 10700 tablet_service.cc:1467] Processing CreateTablet for tablet ae02e32cf9de4757bc930f9b52048f4b (DEFAULT_TABLE table=test-table [id=5dc7ad3051f04cbdb5ddead9a0d5c54e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:48:59.565652 10893 tablet_service.cc:1467] Processing CreateTablet for tablet ae02e32cf9de4757bc930f9b52048f4b (DEFAULT_TABLE table=test-table [id=5dc7ad3051f04cbdb5ddead9a0d5c54e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:48:59.565948 11081 tablet_service.cc:1467] Processing CreateTablet for tablet ae02e32cf9de4757bc930f9b52048f4b (DEFAULT_TABLE table=test-table [id=5dc7ad3051f04cbdb5ddead9a0d5c54e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:48:59.566121 10893 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae02e32cf9de4757bc930f9b52048f4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:48:59.566120 10700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae02e32cf9de4757bc930f9b52048f4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:48:59.566239 11081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae02e32cf9de4757bc930f9b52048f4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:48:59.569592 11205 tablet_bootstrap.cc:492] T ae02e32cf9de4757bc930f9b52048f4b P 2a46ef543d324436a106039cda17bb2a: Bootstrap starting.
I20260806 22:48:59.570106 11208 tablet_bootstrap.cc:492] T ae02e32cf9de4757bc930f9b52048f4b P fc97311847394bab985454d6f623a66b: Bootstrap starting.
I20260806 22:48:59.570364 11206 tablet_bootstrap.cc:492] T ae02e32cf9de4757bc930f9b52048f4b P ebe47a29283a42d7b3bca27f9b064ccb: Bootstrap starting.
I20260806 22:48:59.570555 11205 tablet_bootstrap.cc:654] T ae02e32cf9de4757bc930f9b52048f4b P 2a46ef543d324436a106039cda17bb2a: Neither blocks nor log segments found. Creating new log.
I20260806 22:48:59.571282 11206 tablet_bootstrap.cc:654] T ae02e32cf9de4757bc930f9b52048f4b P ebe47a29283a42d7b3bca27f9b064ccb: Neither blocks nor log segments found. Creating new log.
I20260806 22:48:59.571372 11205 log.cc:824] T ae02e32cf9de4757bc930f9b52048f4b P 2a46ef543d324436a106039cda17bb2a: Log is configured to *not* fsync() on all Append() calls
I20260806 22:48:59.571748 11208 tablet_bootstrap.cc:654] T ae02e32cf9de4757bc930f9b52048f4b P fc97311847394bab985454d6f623a66b: Neither blocks nor log segments found. Creating new log.
I20260806 22:48:59.572010 11206 log.cc:824] T ae02e32cf9de4757bc930f9b52048f4b P ebe47a29283a42d7b3bca27f9b064ccb: Log is configured to *not* fsync() on all Append() calls
I20260806 22:48:59.572329 11205 tablet_bootstrap.cc:492] T ae02e32cf9de4757bc930f9b52048f4b P 2a46ef543d324436a106039cda17bb2a: No bootstrap required, opened a new log
I20260806 22:48:59.572436 11205 ts_tablet_manager.cc:1397] T ae02e32cf9de4757bc930f9b52048f4b P 2a46ef543d324436a106039cda17bb2a: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260806 22:48:59.572513 11208 log.cc:824] T ae02e32cf9de4757bc930f9b52048f4b P fc97311847394bab985454d6f623a66b: Log is configured to *not* fsync() on all Append() calls
I20260806 22:48:59.572903 11206 tablet_bootstrap.cc:492] T ae02e32cf9de4757bc930f9b52048f4b P ebe47a29283a42d7b3bca27f9b064ccb: No bootstrap required, opened a new log
I20260806 22:48:59.573041 11206 ts_tablet_manager.cc:1397] T ae02e32cf9de4757bc930f9b52048f4b P ebe47a29283a42d7b3bca27f9b064ccb: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:48:59.573323 11208 tablet_bootstrap.cc:492] T ae02e32cf9de4757bc930f9b52048f4b P fc97311847394bab985454d6f623a66b: No bootstrap required, opened a new log
I20260806 22:48:59.573408 11208 ts_tablet_manager.cc:1397] T ae02e32cf9de4757bc930f9b52048f4b P fc97311847394bab985454d6f623a66b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:48:59.574708 11205 raft_consensus.cc:348] T ae02e32cf9de4757bc930f9b52048f4b P 2a46ef543d324436a106039cda17bb2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc97311847394bab985454d6f623a66b" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 36433 } } peers { permanent_uuid: "ebe47a29283a42d7b3bca27f9b064ccb" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 36461 } } peers { permanent_uuid: "2a46ef543d324436a106039cda17bb2a" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 33789 } }
I20260806 22:48:59.574944 11205 raft_consensus.cc:374] T ae02e32cf9de4757bc930f9b52048f4b P 2a46ef543d324436a106039cda17bb2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:48:59.575071 11205 raft_consensus.cc:729] T ae02e32cf9de4757bc930f9b52048f4b P 2a46ef543d324436a106039cda17bb2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a46ef543d324436a106039cda17bb2a, State: Initialized, Role: FOLLOWER
I20260806 22:48:59.575239 11205 consensus_queue.cc:260] T ae02e32cf9de4757bc930f9b52048f4b P 2a46ef543d324436a106039cda17bb2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc97311847394bab985454d6f623a66b" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 36433 } } peers { permanent_uuid: "ebe47a29283a42d7b3bca27f9b064ccb" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 36461 } } peers { permanent_uuid: "2a46ef543d324436a106039cda17bb2a" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 33789 } }
I20260806 22:48:59.575295 11206 raft_consensus.cc:348] T ae02e32cf9de4757bc930f9b52048f4b P ebe47a29283a42d7b3bca27f9b064ccb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc97311847394bab985454d6f623a66b" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 36433 } } peers { permanent_uuid: "ebe47a29283a42d7b3bca27f9b064ccb" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 36461 } } peers { permanent_uuid: "2a46ef543d324436a106039cda17bb2a" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 33789 } }
I20260806 22:48:59.575310 11208 raft_consensus.cc:348] T ae02e32cf9de4757bc930f9b52048f4b P fc97311847394bab985454d6f623a66b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc97311847394bab985454d6f623a66b" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 36433 } } peers { permanent_uuid: "ebe47a29283a42d7b3bca27f9b064ccb" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 36461 } } peers { permanent_uuid: "2a46ef543d324436a106039cda17bb2a" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 33789 } }
I20260806 22:48:59.575490 11206 raft_consensus.cc:374] T ae02e32cf9de4757bc930f9b52048f4b P ebe47a29283a42d7b3bca27f9b064ccb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:48:59.575495 11208 raft_consensus.cc:374] T ae02e32cf9de4757bc930f9b52048f4b P fc97311847394bab985454d6f623a66b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:48:59.575536 11208 raft_consensus.cc:729] T ae02e32cf9de4757bc930f9b52048f4b P fc97311847394bab985454d6f623a66b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc97311847394bab985454d6f623a66b, State: Initialized, Role: FOLLOWER
I20260806 22:48:59.575592 11205 ts_tablet_manager.cc:1428] T ae02e32cf9de4757bc930f9b52048f4b P 2a46ef543d324436a106039cda17bb2a: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 22:48:59.575584 11206 raft_consensus.cc:729] T ae02e32cf9de4757bc930f9b52048f4b P ebe47a29283a42d7b3bca27f9b064ccb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebe47a29283a42d7b3bca27f9b064ccb, State: Initialized, Role: FOLLOWER
I20260806 22:48:59.575650 11208 consensus_queue.cc:260] T ae02e32cf9de4757bc930f9b52048f4b P fc97311847394bab985454d6f623a66b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc97311847394bab985454d6f623a66b" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 36433 } } peers { permanent_uuid: "ebe47a29283a42d7b3bca27f9b064ccb" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 36461 } } peers { permanent_uuid: "2a46ef543d324436a106039cda17bb2a" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 33789 } }
I20260806 22:48:59.575718 11206 consensus_queue.cc:260] T ae02e32cf9de4757bc930f9b52048f4b P ebe47a29283a42d7b3bca27f9b064ccb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc97311847394bab985454d6f623a66b" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 36433 } } peers { permanent_uuid: "ebe47a29283a42d7b3bca27f9b064ccb" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 36461 } } peers { permanent_uuid: "2a46ef543d324436a106039cda17bb2a" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 33789 } }
I20260806 22:48:59.575793 11181 heartbeater.cc:503] Master 127.18.45.254:36079 was elected leader, sending a full tablet report...
I20260806 22:48:59.575961 11208 ts_tablet_manager.cc:1428] T ae02e32cf9de4757bc930f9b52048f4b P fc97311847394bab985454d6f623a66b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:48:59.576000 10796 heartbeater.cc:503] Master 127.18.45.254:36079 was elected leader, sending a full tablet report...
I20260806 22:48:59.576032 11206 ts_tablet_manager.cc:1428] T ae02e32cf9de4757bc930f9b52048f4b P ebe47a29283a42d7b3bca27f9b064ccb: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:48:59.576176 10988 heartbeater.cc:503] Master 127.18.45.254:36079 was elected leader, sending a full tablet report...
W20260806 22:48:59.590134 10989 tablet.cc:2369] T ae02e32cf9de4757bc930f9b52048f4b P ebe47a29283a42d7b3bca27f9b064ccb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:48:59.635960 11212 raft_consensus.cc:482] T ae02e32cf9de4757bc930f9b52048f4b P 2a46ef543d324436a106039cda17bb2a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:48:59.636147 11212 raft_consensus.cc:504] T ae02e32cf9de4757bc930f9b52048f4b P 2a46ef543d324436a106039cda17bb2a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc97311847394bab985454d6f623a66b" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 36433 } } peers { permanent_uuid: "ebe47a29283a42d7b3bca27f9b064ccb" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 36461 } } peers { permanent_uuid: "2a46ef543d324436a106039cda17bb2a" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 33789 } }
I20260806 22:48:59.636683 11212 leader_election.cc:288] T ae02e32cf9de4757bc930f9b52048f4b P 2a46ef543d324436a106039cda17bb2a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fc97311847394bab985454d6f623a66b (127.18.45.193:36433), ebe47a29283a42d7b3bca27f9b064ccb (127.18.45.194:36461)
I20260806 22:48:59.641340 11204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:59.637329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.195:45563 (local address 127.18.45.194:36461)
0806 22:48:59.637401 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 22:48:59.637403 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:48:59.637543 (+ 140us) server_negotiation.cc:407] Connection header received
0806 22:48:59.637593 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:48:59.637594 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:48:59.637604 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:48:59.637661 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:48:59.637848 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:59.638588 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:59.640286 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:59.640917 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:59.640997 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:59.641006 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:48:59.641197 (+ 191us) server_negotiation.cc:299] Negotiation successful
0806 22:48:59.641253 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 22:48:59.641340 11200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:59.637193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.195:52173 (local address 127.18.45.193:36433)
0806 22:48:59.637307 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0806 22:48:59.637310 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:48:59.637336 (+ 26us) server_negotiation.cc:407] Connection header received
0806 22:48:59.637382 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:48:59.637384 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:48:59.637398 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:48:59.637471 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:48:59.637724 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:59.638580 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:59.640286 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:59.640899 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:59.640992 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:59.641006 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0806 22:48:59.641197 (+ 191us) server_negotiation.cc:299] Negotiation successful
0806 22:48:59.641250 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 22:48:59.641350 11186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:59.637049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.193:36433 (local address 127.18.45.195:52173)
0806 22:48:59.637149 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:59.637166 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 22:48:59.637257 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:48:59.637504 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:48:59.637505 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:48:59.637507 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:48:59.637612 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:48:59.637614 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:48:59.638705 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:48:59.638706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:48:59.640159 (+ 1453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:48:59.640164 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:48:59.641011 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:48:59.641013 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:48:59.641113 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:59.641115 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 22:48:59.641174 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 22:48:59.641231 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64,"mutex_wait_us":70}
I20260806 22:48:59.641361 11217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:59.637205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.194:36461 (local address 127.18.45.195:45563)
0806 22:48:59.637439 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:59.637451 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:48:59.637526 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:48:59.637679 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:48:59.637680 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:48:59.637682 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:48:59.637767 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:48:59.637769 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:48:59.638705 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:48:59.638706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:48:59.640159 (+ 1453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:48:59.640164 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:48:59.641033 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:48:59.641034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:48:59.641107 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:59.641112 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:48:59.641174 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 22:48:59.641231 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"mutex_wait_us":82,"thread_start_us":151,"threads_started":1}
I20260806 22:48:59.641867 10921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae02e32cf9de4757bc930f9b52048f4b" candidate_uuid: "2a46ef543d324436a106039cda17bb2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebe47a29283a42d7b3bca27f9b064ccb" is_pre_election: true
I20260806 22:48:59.641865 10725 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae02e32cf9de4757bc930f9b52048f4b" candidate_uuid: "2a46ef543d324436a106039cda17bb2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc97311847394bab985454d6f623a66b" is_pre_election: true
I20260806 22:48:59.642086 10725 raft_consensus.cc:2454] T ae02e32cf9de4757bc930f9b52048f4b P fc97311847394bab985454d6f623a66b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a46ef543d324436a106039cda17bb2a in term 0.
I20260806 22:48:59.642089 10921 raft_consensus.cc:2454] T ae02e32cf9de4757bc930f9b52048f4b P ebe47a29283a42d7b3bca27f9b064ccb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a46ef543d324436a106039cda17bb2a in term 0.
I20260806 22:48:59.642393 11018 leader_election.cc:302] T ae02e32cf9de4757bc930f9b52048f4b P 2a46ef543d324436a106039cda17bb2a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a46ef543d324436a106039cda17bb2a, fc97311847394bab985454d6f623a66b; no voters:
I20260806 22:48:59.642541 11212 raft_consensus.cc:2781] T ae02e32cf9de4757bc930f9b52048f4b P 2a46ef543d324436a106039cda17bb2a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:48:59.642606 11212 raft_consensus.cc:482] T ae02e32cf9de4757bc930f9b52048f4b P 2a46ef543d324436a106039cda17bb2a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:48:59.642622 11212 raft_consensus.cc:3037] T ae02e32cf9de4757bc930f9b52048f4b P 2a46ef543d324436a106039cda17bb2a [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:48:59.643550 11212 raft_consensus.cc:504] T ae02e32cf9de4757bc930f9b52048f4b P 2a46ef543d324436a106039cda17bb2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc97311847394bab985454d6f623a66b" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 36433 } } peers { permanent_uuid: "ebe47a29283a42d7b3bca27f9b064ccb" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 36461 } } peers { permanent_uuid: "2a46ef543d324436a106039cda17bb2a" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 33789 } }
I20260806 22:48:59.643682 11212 leader_election.cc:288] T ae02e32cf9de4757bc930f9b52048f4b P 2a46ef543d324436a106039cda17bb2a [CANDIDATE]: Term 1 election: Requested vote from peers fc97311847394bab985454d6f623a66b (127.18.45.193:36433), ebe47a29283a42d7b3bca27f9b064ccb (127.18.45.194:36461)
I20260806 22:48:59.644119 10725 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae02e32cf9de4757bc930f9b52048f4b" candidate_uuid: "2a46ef543d324436a106039cda17bb2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc97311847394bab985454d6f623a66b"
I20260806 22:48:59.644121 10921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ae02e32cf9de4757bc930f9b52048f4b" candidate_uuid: "2a46ef543d324436a106039cda17bb2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebe47a29283a42d7b3bca27f9b064ccb"
I20260806 22:48:59.644196 10725 raft_consensus.cc:3037] T ae02e32cf9de4757bc930f9b52048f4b P fc97311847394bab985454d6f623a66b [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:48:59.644223 10921 raft_consensus.cc:3037] T ae02e32cf9de4757bc930f9b52048f4b P ebe47a29283a42d7b3bca27f9b064ccb [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:48:59.645238 10725 raft_consensus.cc:2454] T ae02e32cf9de4757bc930f9b52048f4b P fc97311847394bab985454d6f623a66b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a46ef543d324436a106039cda17bb2a in term 1.
I20260806 22:48:59.645298 10921 raft_consensus.cc:2454] T ae02e32cf9de4757bc930f9b52048f4b P ebe47a29283a42d7b3bca27f9b064ccb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a46ef543d324436a106039cda17bb2a in term 1.
I20260806 22:48:59.645462 11018 leader_election.cc:302] T ae02e32cf9de4757bc930f9b52048f4b P 2a46ef543d324436a106039cda17bb2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a46ef543d324436a106039cda17bb2a, fc97311847394bab985454d6f623a66b; no voters:
I20260806 22:48:59.645578 11212 raft_consensus.cc:2781] T ae02e32cf9de4757bc930f9b52048f4b P 2a46ef543d324436a106039cda17bb2a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:48:59.645799 11212 raft_consensus.cc:686] T ae02e32cf9de4757bc930f9b52048f4b P 2a46ef543d324436a106039cda17bb2a [term 1 LEADER]: Becoming Leader. State: Replica: 2a46ef543d324436a106039cda17bb2a, State: Running, Role: LEADER
I20260806 22:48:59.645956 11212 consensus_queue.cc:237] T ae02e32cf9de4757bc930f9b52048f4b P 2a46ef543d324436a106039cda17bb2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc97311847394bab985454d6f623a66b" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 36433 } } peers { permanent_uuid: "ebe47a29283a42d7b3bca27f9b064ccb" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 36461 } } peers { permanent_uuid: "2a46ef543d324436a106039cda17bb2a" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 33789 } }
I20260806 22:48:59.647153 10527 catalog_manager.cc:5495] T ae02e32cf9de4757bc930f9b52048f4b P 2a46ef543d324436a106039cda17bb2a reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a46ef543d324436a106039cda17bb2a (127.18.45.195). New cstate: current_term: 1 leader_uuid: "2a46ef543d324436a106039cda17bb2a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc97311847394bab985454d6f623a66b" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 36433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ebe47a29283a42d7b3bca27f9b064ccb" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 36461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a46ef543d324436a106039cda17bb2a" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 33789 } health_report { overall_health: HEALTHY } } }
W20260806 22:48:59.649720 10798 tablet.cc:2369] T ae02e32cf9de4757bc930f9b52048f4b P fc97311847394bab985454d6f623a66b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:48:59.657291 11202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:59.654292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39260 (local address 127.18.45.195:33789)
0806 22:48:59.654379 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0806 22:48:59.654382 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:48:59.654392 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:48:59.654442 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:48:59.654443 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:48:59.654475 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:48:59.654579 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:48:59.654765 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:59.655445 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:59.656335 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:59.656570 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:59.656669 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:59.656874 (+ 205us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:48:59.657015 (+ 141us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:48:59.657124 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 22:48:59.657147 (+ 23us) server_negotiation.cc:299] Negotiation successful
0806 22:48:59.657172 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 22:48:59.659788 10921 raft_consensus.cc:1261] T ae02e32cf9de4757bc930f9b52048f4b P ebe47a29283a42d7b3bca27f9b064ccb [term 1 FOLLOWER]: Refusing update from remote peer 2a46ef543d324436a106039cda17bb2a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:48:59.659786 10725 raft_consensus.cc:1261] T ae02e32cf9de4757bc930f9b52048f4b P fc97311847394bab985454d6f623a66b [term 1 FOLLOWER]: Refusing update from remote peer 2a46ef543d324436a106039cda17bb2a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:48:59.660261 11218 consensus_queue.cc:1035] T ae02e32cf9de4757bc930f9b52048f4b P 2a46ef543d324436a106039cda17bb2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc97311847394bab985454d6f623a66b" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 36433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:48:59.660444 11212 consensus_queue.cc:1035] T ae02e32cf9de4757bc930f9b52048f4b P 2a46ef543d324436a106039cda17bb2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ebe47a29283a42d7b3bca27f9b064ccb" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 36461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:48:59.663832 11226 mvcc.cc:205] Tried to move back new op lower bound from 7315687586442670080 to 7315687586390777856. Current Snapshot: MvccSnapshot[applied={T|T < 7315687586442670080}]
I20260806 22:48:59.664268 11225 mvcc.cc:205] Tried to move back new op lower bound from 7315687586442670080 to 7315687586390777856. Current Snapshot: MvccSnapshot[applied={T|T < 7315687586442670080}]
I20260806 22:48:59.666483 10529 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38108:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:48:59.666605 10529 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38108:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:48:59.667654 10529 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c5aad1fbc4be4a338b4e868af0b7ff66: Sending DeleteTablet for 3 replicas of tablet ae02e32cf9de4757bc930f9b52048f4b
I20260806 22:48:59.668076 10700 tablet_service.cc:1514] Processing DeleteTablet for tablet ae02e32cf9de4757bc930f9b52048f4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:48:59 UTC) from {username='kudu', principal='kudu/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:34550
I20260806 22:48:59.668120 10893 tablet_service.cc:1514] Processing DeleteTablet for tablet ae02e32cf9de4757bc930f9b52048f4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:48:59 UTC) from {username='kudu', principal='kudu/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:41608
I20260806 22:48:59.668116 11081 tablet_service.cc:1514] Processing DeleteTablet for tablet ae02e32cf9de4757bc930f9b52048f4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:48:59 UTC) from {username='kudu', principal='kudu/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:39252
I20260806 22:48:59.668412 11242 tablet_replica.cc:330] T ae02e32cf9de4757bc930f9b52048f4b P 2a46ef543d324436a106039cda17bb2a: stopping tablet replica
I20260806 22:48:59.668540 11242 raft_consensus.cc:2229] T ae02e32cf9de4757bc930f9b52048f4b P 2a46ef543d324436a106039cda17bb2a [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:48:59.668645 11242 raft_consensus.cc:2258] T ae02e32cf9de4757bc930f9b52048f4b P 2a46ef543d324436a106039cda17bb2a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:48:59.668761 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 10610
I20260806 22:48:59.676748 11242 ts_tablet_manager.cc:1905] T ae02e32cf9de4757bc930f9b52048f4b P 2a46ef543d324436a106039cda17bb2a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:48:59.679549 11242 ts_tablet_manager.cc:1918] T ae02e32cf9de4757bc930f9b52048f4b P 2a46ef543d324436a106039cda17bb2a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:48:59.679690 11242 log.cc:1196] T ae02e32cf9de4757bc930f9b52048f4b P 2a46ef543d324436a106039cda17bb2a: Deleting WAL directory at /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/minicluster-data/ts-2/wal/wals/ae02e32cf9de4757bc930f9b52048f4b
I20260806 22:48:59.679932 11239 tablet_replica.cc:330] T ae02e32cf9de4757bc930f9b52048f4b P ebe47a29283a42d7b3bca27f9b064ccb: stopping tablet replica
I20260806 22:48:59.680022 11242 ts_tablet_manager.cc:1939] T ae02e32cf9de4757bc930f9b52048f4b P 2a46ef543d324436a106039cda17bb2a: Deleting consensus metadata
I20260806 22:48:59.680212 11239 raft_consensus.cc:2229] T ae02e32cf9de4757bc930f9b52048f4b P ebe47a29283a42d7b3bca27f9b064ccb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:48:59.680414 11239 raft_consensus.cc:2258] T ae02e32cf9de4757bc930f9b52048f4b P ebe47a29283a42d7b3bca27f9b064ccb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:48:59.680589 10512 catalog_manager.cc:4841] TS 2a46ef543d324436a106039cda17bb2a (127.18.45.195:33789): tablet ae02e32cf9de4757bc930f9b52048f4b (table test-table [id=5dc7ad3051f04cbdb5ddead9a0d5c54e]) successfully deleted
I20260806 22:48:59.681792 11239 ts_tablet_manager.cc:1905] T ae02e32cf9de4757bc930f9b52048f4b P ebe47a29283a42d7b3bca27f9b064ccb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:48:59.683792 11239 ts_tablet_manager.cc:1918] T ae02e32cf9de4757bc930f9b52048f4b P ebe47a29283a42d7b3bca27f9b064ccb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:48:59.683887 11239 log.cc:1196] T ae02e32cf9de4757bc930f9b52048f4b P ebe47a29283a42d7b3bca27f9b064ccb: Deleting WAL directory at /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786056275803658-18615-0/minicluster-data/ts-1/wal/wals/ae02e32cf9de4757bc930f9b52048f4b
I20260806 22:48:59.684206 11239 ts_tablet_manager.cc:1939] T ae02e32cf9de4757bc930f9b52048f4b P ebe47a29283a42d7b3bca27f9b064ccb: Deleting consensus metadata
I20260806 22:48:59.684793 10510 catalog_manager.cc:4841] TS ebe47a29283a42d7b3bca27f9b064ccb (127.18.45.194:36461): tablet ae02e32cf9de4757bc930f9b52048f4b (table test-table [id=5dc7ad3051f04cbdb5ddead9a0d5c54e]) successfully deleted
W20260806 22:48:59.688230 10510 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.45.193:36433 (error 108)
I20260806 22:48:59.688544 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 10803
I20260806 22:48:59.692679 11197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:59.692428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.193:36433 (local address 127.0.0.1:34556)
0806 22:48:59.692528 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0806 22:48:59.692615 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.193:36433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260806 22:48:59.693096 10510 catalog_manager.cc:4584] TS fc97311847394bab985454d6f623a66b (127.18.45.193:36433): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ae02e32cf9de4757bc930f9b52048f4b: Network error: Client connection negotiation failed: client connection to 127.18.45.193:36433: connect: Connection refused (error 111)
I20260806 22:48:59.706322 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 10993
I20260806 22:48:59.721029 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 10488
2026-08-06T22:48:59Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2394 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-06T22:48:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:48:59Z Disabled control of system clock
I20260806 22:48:59.764142 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056275803658-18615-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056275803658-18615-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056275803658-18615-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:42581
--webserver_interface=127.18.45.254
--webserver_port=0
--builtin_ntp_servers=127.18.45.212:45163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.45.254:42581
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 22:48:59.857501 11249 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:48:59.857800 11249 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:48:59.857822 11249 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:48:59.859026 11249 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:48:59.859114 11249 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:48:59.859136 11249 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:48:59.859156 11249 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:48:59.859189 11249 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:48:59.860663 11249 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:45163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056275803658-18615-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056275803658-18615-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.45.254:42581
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:42581
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.45.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11249
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056275803658-18615-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:48:59.861004 11249 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:48:59.861361 11249 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:48:59.867128 11259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:48:59.867113 11258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:48:59.867115 11263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:48:59.867394 11249 server_base.cc:1034] running on GCE node
I20260806 22:48:59.867702 11249 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:48:59.868060 11249 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:48:59.869233 11249 hybrid_clock.cc:648] HybridClock initialized: now 1786056539869226 us; error 39 us; skew 500 ppm
I20260806 22:48:59.870638 11249 webserver.cc:460] Webserver started at http://127.18.45.254:34817/ using document root <none> and password file <none>
I20260806 22:48:59.870918 11249 fs_manager.cc:359] Metadata directory not provided
I20260806 22:48:59.871001 11249 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:48:59.871191 11249 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:48:59.872432 11249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056275803658-18615-0/minicluster-data/master-0/data/instance:
uuid: "003fbbf432c64e139279c18ba531a6f9"
format_stamp: "Formatted at 2026-08-06 22:48:59 on dist-test-slave-h3ft"
server_key: "fa56f6cbfafded994a34d0dabf5bdf4e"
server_key_iv: "35a811bee737d2fd95e8ab85cd1d3e6b"
server_key_version: "encryptionkey@0"
I20260806 22:48:59.872907 11249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056275803658-18615-0/minicluster-data/master-0/wal/instance:
uuid: "003fbbf432c64e139279c18ba531a6f9"
format_stamp: "Formatted at 2026-08-06 22:48:59 on dist-test-slave-h3ft"
server_key: "fa56f6cbfafded994a34d0dabf5bdf4e"
server_key_iv: "35a811bee737d2fd95e8ab85cd1d3e6b"
server_key_version: "encryptionkey@0"
I20260806 22:48:59.874423 11249 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:48:59.875317 11270 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:48:59.875504 11249 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:48:59.875622 11249 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056275803658-18615-0/minicluster-data/master-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056275803658-18615-0/minicluster-data/master-0/wal
uuid: "003fbbf432c64e139279c18ba531a6f9"
format_stamp: "Formatted at 2026-08-06 22:48:59 on dist-test-slave-h3ft"
server_key: "fa56f6cbfafded994a34d0dabf5bdf4e"
server_key_iv: "35a811bee737d2fd95e8ab85cd1d3e6b"
server_key_version: "encryptionkey@0"
I20260806 22:48:59.875705 11249 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056275803658-18615-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056275803658-18615-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056275803658-18615-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:48:59.896879 11249 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:48:59.897370 11249 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:48:59.897578 11249 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:48:59.902987 11249 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.254:42581
I20260806 22:48:59.903046 11343 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.254:42581 every 8 connection(s)
I20260806 22:48:59.903576 11249 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
I20260806 22:48:59.904866 11344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:48:59.907713 11344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 003fbbf432c64e139279c18ba531a6f9: Bootstrap starting.
I20260806 22:48:59.908390 11344 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 003fbbf432c64e139279c18ba531a6f9: Neither blocks nor log segments found. Creating new log.
I20260806 22:48:59.908684 11344 log.cc:824] T 00000000000000000000000000000000 P 003fbbf432c64e139279c18ba531a6f9: Log is configured to *not* fsync() on all Append() calls
I20260806 22:48:59.909441 11344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 003fbbf432c64e139279c18ba531a6f9: No bootstrap required, opened a new log
I20260806 22:48:59.910032 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 11249
I20260806 22:48:59.910161 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056275803658-18615-0/minicluster-data/master-0/wal/instance
I20260806 22:48:59.910403 18615 external_mini_cluster.cc:1428] Setting key d07cdce1d0d7c7b3601efaf09571f564
I20260806 22:48:59.911918 11344 raft_consensus.cc:348] T 00000000000000000000000000000000 P 003fbbf432c64e139279c18ba531a6f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "003fbbf432c64e139279c18ba531a6f9" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 42581 } }
I20260806 22:48:59.912086 11344 raft_consensus.cc:374] T 00000000000000000000000000000000 P 003fbbf432c64e139279c18ba531a6f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:48:59.912110 11344 raft_consensus.cc:729] T 00000000000000000000000000000000 P 003fbbf432c64e139279c18ba531a6f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 003fbbf432c64e139279c18ba531a6f9, State: Initialized, Role: FOLLOWER
I20260806 22:48:59.912284 11344 consensus_queue.cc:260] T 00000000000000000000000000000000 P 003fbbf432c64e139279c18ba531a6f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "003fbbf432c64e139279c18ba531a6f9" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 42581 } }
I20260806 22:48:59.912374 11344 raft_consensus.cc:388] T 00000000000000000000000000000000 P 003fbbf432c64e139279c18ba531a6f9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:48:59.912400 11344 raft_consensus.cc:482] T 00000000000000000000000000000000 P 003fbbf432c64e139279c18ba531a6f9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:48:59.912465 11344 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 003fbbf432c64e139279c18ba531a6f9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:48:59.913087 11344 raft_consensus.cc:504] T 00000000000000000000000000000000 P 003fbbf432c64e139279c18ba531a6f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "003fbbf432c64e139279c18ba531a6f9" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 42581 } }
I20260806 22:48:59.913213 11344 leader_election.cc:302] T 00000000000000000000000000000000 P 003fbbf432c64e139279c18ba531a6f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 003fbbf432c64e139279c18ba531a6f9; no voters:
I20260806 22:48:59.913405 11344 leader_election.cc:288] T 00000000000000000000000000000000 P 003fbbf432c64e139279c18ba531a6f9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:48:59.913439 11352 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 003fbbf432c64e139279c18ba531a6f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:48:59.913581 11352 raft_consensus.cc:686] T 00000000000000000000000000000000 P 003fbbf432c64e139279c18ba531a6f9 [term 1 LEADER]: Becoming Leader. State: Replica: 003fbbf432c64e139279c18ba531a6f9, State: Running, Role: LEADER
I20260806 22:48:59.913726 11344 sys_catalog.cc:564] T 00000000000000000000000000000000 P 003fbbf432c64e139279c18ba531a6f9 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:48:59.913688 11352 consensus_queue.cc:237] T 00000000000000000000000000000000 P 003fbbf432c64e139279c18ba531a6f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "003fbbf432c64e139279c18ba531a6f9" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 42581 } }
I20260806 22:48:59.914199 11356 sys_catalog.cc:455] T 00000000000000000000000000000000 P 003fbbf432c64e139279c18ba531a6f9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 003fbbf432c64e139279c18ba531a6f9. Latest consensus state: current_term: 1 leader_uuid: "003fbbf432c64e139279c18ba531a6f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "003fbbf432c64e139279c18ba531a6f9" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 42581 } } }
I20260806 22:48:59.914382 11356 sys_catalog.cc:458] T 00000000000000000000000000000000 P 003fbbf432c64e139279c18ba531a6f9 [sys.catalog]: This master's current role is: LEADER
I20260806 22:48:59.914618 11355 sys_catalog.cc:455] T 00000000000000000000000000000000 P 003fbbf432c64e139279c18ba531a6f9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "003fbbf432c64e139279c18ba531a6f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "003fbbf432c64e139279c18ba531a6f9" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 42581 } } }
I20260806 22:48:59.914740 11355 sys_catalog.cc:458] T 00000000000000000000000000000000 P 003fbbf432c64e139279c18ba531a6f9 [sys.catalog]: This master's current role is: LEADER
I20260806 22:48:59.915217 11371 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:48:59.915809 11371 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:48:59.917439 11371 catalog_manager.cc:1347] Generated new cluster ID: e9a95e83ca3449639980995a5d9b7094
I20260806 22:48:59.917498 11371 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:48:59.918231 11351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:48:59.911133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34322 (local address 127.18.45.254:42581)
0806 22:48:59.911352 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0806 22:48:59.911361 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:48:59.911390 (+ 29us) server_negotiation.cc:407] Connection header received
0806 22:48:59.911929 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:48:59.911938 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:48:59.912015 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:48:59.912218 (+ 203us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:48:59.912537 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:59.914260 (+ 1723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:59.915566 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:48:59.916236 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:48:59.916749 (+ 513us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:48:59.917274 (+ 525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:48:59.917278 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:48:59.917279 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:48:59.917518 (+ 239us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:48:59.917564 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:48:59.917580 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:48:59.917652 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 22:48:59.917769 (+ 117us) server_negotiation.cc:299] Negotiation successful
0806 22:48:59.917988 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":92,"threads_started":1}
I20260806 22:48:59.934657 11371 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:48:59.935521 11371 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:48:59.943949 11371 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 003fbbf432c64e139279c18ba531a6f9: Generated new TSK 0
I20260806 22:48:59.944226 11371 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:48:59.952347 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056275803658-18615-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056275803658-18615-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056275803658-18615-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.193:0
--local_ip_for_outbound_sockets=127.18.45.193
--webserver_interface=127.18.45.193
--webserver_port=0
--tserver_master_addrs=127.18.45.254:42581
--builtin_ntp_servers=127.18.45.212:45163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 22:49:00.057938 11384 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:00.058238 11384 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:00.058281 11384 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:00.059643 11384 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:00.059719 11384 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:00.059763 11384 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.193
I20260806 22:49:00.061291 11384 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:45163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056275803658-18615-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056275803658-18615-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.45.193
--webserver_port=0
--tserver_master_addrs=127.18.45.254:42581
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11384
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.193
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056275803658-18615-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:49:00.061612 11384 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:00.062018 11384 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:00.069643 11390 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:00.069618 11393 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:00.069603 11391 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:00.069959 11384 server_base.cc:1034] running on GCE node
I20260806 22:49:00.070222 11384 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:00.070560 11384 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:00.071762 11384 hybrid_clock.cc:648] HybridClock initialized: now 1786056540071714 us; error 58 us; skew 500 ppm
I20260806 22:49:00.073484 11384 webserver.cc:460] Webserver started at http://127.18.45.193:43655/ using document root <none> and password file <none>
I20260806 22:49:00.073822 11384 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:00.073954 11384 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:00.074139 11384 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:49:00.075533 11384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056275803658-18615-0/minicluster-data/ts-0/data/instance:
uuid: "a7bf7e05910e4350bfab1ba0d220d85e"
format_stamp: "Formatted at 2026-08-06 22:49:00 on dist-test-slave-h3ft"
server_key: "61b90a7fc0261c25cacb2ed889ab300f"
server_key_iv: "a93d8998152936852b33614640b47e39"
server_key_version: "encryptionkey@0"
I20260806 22:49:00.076009 11384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056275803658-18615-0/minicluster-data/ts-0/wal/instance:
uuid: "a7bf7e05910e4350bfab1ba0d220d85e"
format_stamp: "Formatted at 2026-08-06 22:49:00 on dist-test-slave-h3ft"
server_key: "61b90a7fc0261c25cacb2ed889ab300f"
server_key_iv: "a93d8998152936852b33614640b47e39"
server_key_version: "encryptionkey@0"
I20260806 22:49:00.077567 11384 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:49:00.078696 11411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:00.079078 11384 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:49:00.079205 11384 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056275803658-18615-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056275803658-18615-0/minicluster-data/ts-0/wal
uuid: "a7bf7e05910e4350bfab1ba0d220d85e"
format_stamp: "Formatted at 2026-08-06 22:49:00 on dist-test-slave-h3ft"
server_key: "61b90a7fc0261c25cacb2ed889ab300f"
server_key_iv: "a93d8998152936852b33614640b47e39"
server_key_version: "encryptionkey@0"
I20260806 22:49:00.079304 11384 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056275803658-18615-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056275803658-18615-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056275803658-18615-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:00.101578 11384 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:00.102185 11384 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:00.102461 11384 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:00.102828 11384 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:49:00.103446 11384 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:49:00.103518 11384 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:00.103564 11384 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:49:00.103619 11384 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:00.111765 11384 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.193:36637
I20260806 22:49:00.111824 11569 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.193:36637 every 8 connection(s)
I20260806 22:49:00.112264 11384 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
I20260806 22:49:00.119110 11351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:00.113265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:59875 (local address 127.18.45.254:42581)
0806 22:49:00.113355 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:00.113357 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:00.113656 (+ 299us) server_negotiation.cc:407] Connection header received
0806 22:49:00.114297 (+ 641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:00.114299 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:00.114317 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:00.114422 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:00.115559 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:00.116192 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:00.117391 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:00.117608 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:00.117664 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:00.118698 (+ 1034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:00.118700 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:00.118700 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:49:00.118707 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:00.118727 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:49:00.118732 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:00.118779 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:00.118937 (+ 158us) server_negotiation.cc:299] Negotiation successful
0806 22:49:00.118968 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 22:49:00.119392 11573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:00.113194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:42581 (local address 127.18.45.193:59875)
0806 22:49:00.113487 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:00.113550 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:00.114193 (+ 643us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:00.114577 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:00.114581 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:00.114595 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:00.115425 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:00.115434 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:00.116293 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:00.116294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:00.117295 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:00.117298 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:00.117695 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:00.117696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:00.117836 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:00.118533 (+ 697us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:49:00.118534 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:00.118571 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:49:00.118603 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:00.118822 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:00.118830 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:00.118833 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:49:00.118928 (+ 95us) client_negotiation.cc:240] Negotiation successful
0806 22:49:00.119115 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":121,"threads_started":1}
I20260806 22:49:00.120126 11571 heartbeater.cc:347] Connected to a master server at 127.18.45.254:42581
I20260806 22:49:00.120275 11571 heartbeater.cc:464] Registering TS with master...
I20260806 22:49:00.120515 11571 heartbeater.cc:511] Master 127.18.45.254:42581 requested a full tablet report, sending...
I20260806 22:49:00.121179 11292 ts_manager.cc:194] Registered new tserver with Master: a7bf7e05910e4350bfab1ba0d220d85e (127.18.45.193:36637)
I20260806 22:49:00.121809 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 11384
I20260806 22:49:00.121922 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056275803658-18615-0/minicluster-data/ts-0/wal/instance
I20260806 22:49:00.122193 18615 external_mini_cluster.cc:1428] Setting key 4b932055ea0c360fe0e104f2a3811a25
I20260806 22:49:00.122215 11292 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.45.193:59875
I20260806 22:49:00.123399 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056275803658-18615-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056275803658-18615-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056275803658-18615-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.194:0
--local_ip_for_outbound_sockets=127.18.45.194
--webserver_interface=127.18.45.194
--webserver_port=0
--tserver_master_addrs=127.18.45.254:42581
--builtin_ntp_servers=127.18.45.212:45163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 22:49:00.218444 11574 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:00.218695 11574 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:00.218719 11574 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:00.220038 11574 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:00.220084 11574 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:00.220110 11574 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.194
I20260806 22:49:00.221551 11574 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:45163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056275803658-18615-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056275803658-18615-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.45.194
--webserver_port=0
--tserver_master_addrs=127.18.45.254:42581
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11574
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.194
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056275803658-18615-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:49:00.221863 11574 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:00.222190 11574 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:00.227842 11586 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:00.228012 11585 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:00.228170 11588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:00.228440 11574 server_base.cc:1034] running on GCE node
I20260806 22:49:00.228641 11574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:00.228865 11574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:00.230033 11574 hybrid_clock.cc:648] HybridClock initialized: now 1786056540230017 us; error 36 us; skew 500 ppm
I20260806 22:49:00.231587 11574 webserver.cc:460] Webserver started at http://127.18.45.194:46245/ using document root <none> and password file <none>
I20260806 22:49:00.231895 11574 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:00.231986 11574 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:00.232196 11574 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:49:00.233451 11574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056275803658-18615-0/minicluster-data/ts-1/data/instance:
uuid: "e6e3e92928664ddcb07260a9e023ed9c"
format_stamp: "Formatted at 2026-08-06 22:49:00 on dist-test-slave-h3ft"
server_key: "4c4ee781bbc7f47ec269c54e7971ec37"
server_key_iv: "1c47e639b7201eb1472974ca6b3184d4"
server_key_version: "encryptionkey@0"
I20260806 22:49:00.233851 11574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056275803658-18615-0/minicluster-data/ts-1/wal/instance:
uuid: "e6e3e92928664ddcb07260a9e023ed9c"
format_stamp: "Formatted at 2026-08-06 22:49:00 on dist-test-slave-h3ft"
server_key: "4c4ee781bbc7f47ec269c54e7971ec37"
server_key_iv: "1c47e639b7201eb1472974ca6b3184d4"
server_key_version: "encryptionkey@0"
I20260806 22:49:00.235378 11574 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260806 22:49:00.236238 11598 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:00.236433 11574 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:49:00.236557 11574 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056275803658-18615-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056275803658-18615-0/minicluster-data/ts-1/wal
uuid: "e6e3e92928664ddcb07260a9e023ed9c"
format_stamp: "Formatted at 2026-08-06 22:49:00 on dist-test-slave-h3ft"
server_key: "4c4ee781bbc7f47ec269c54e7971ec37"
server_key_iv: "1c47e639b7201eb1472974ca6b3184d4"
server_key_version: "encryptionkey@0"
I20260806 22:49:00.236639 11574 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056275803658-18615-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056275803658-18615-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056275803658-18615-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:00.251379 11574 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:00.251782 11574 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:00.251950 11574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:00.252239 11574 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:49:00.252658 11574 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:49:00.252719 11574 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:00.252769 11574 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:49:00.252802 11574 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:00.260231 11574 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.194:34907
I20260806 22:49:00.260298 11738 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.194:34907 every 8 connection(s)
I20260806 22:49:00.260703 11574 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
I20260806 22:49:00.266981 11351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:00.261584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:55827 (local address 127.18.45.254:42581)
0806 22:49:00.261671 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:00.261673 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:00.261991 (+ 318us) server_negotiation.cc:407] Connection header received
0806 22:49:00.262572 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:00.262573 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:00.262585 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:00.262655 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:00.263653 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:00.264352 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:00.265445 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:00.265644 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:00.265704 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:00.266707 (+ 1003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:00.266709 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:00.266709 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:49:00.266715 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:00.266729 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:49:00.266733 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:00.266777 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:00.266869 (+ 92us) server_negotiation.cc:299] Negotiation successful
0806 22:49:00.266897 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 22:49:00.267318 11742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:00.261556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:42581 (local address 127.18.45.194:55827)
0806 22:49:00.261806 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:00.261858 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:00.262465 (+ 607us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:00.262744 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:00.262749 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:00.262759 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:00.263555 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:00.263562 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:00.264443 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:00.264444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:00.265356 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:00.265359 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:00.265728 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:00.265729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:00.265858 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:00.266558 (+ 700us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:49:00.266559 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:00.266593 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:49:00.266622 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:00.266793 (+ 171us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:00.266798 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:00.266801 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:49:00.266858 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 22:49:00.266957 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":134,"threads_started":1}
I20260806 22:49:00.267951 11739 heartbeater.cc:347] Connected to a master server at 127.18.45.254:42581
I20260806 22:49:00.268054 11739 heartbeater.cc:464] Registering TS with master...
I20260806 22:49:00.268239 11739 heartbeater.cc:511] Master 127.18.45.254:42581 requested a full tablet report, sending...
I20260806 22:49:00.268680 11292 ts_manager.cc:194] Registered new tserver with Master: e6e3e92928664ddcb07260a9e023ed9c (127.18.45.194:34907)
I20260806 22:49:00.269119 11292 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.45.194:55827
I20260806 22:49:00.270810 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 11574
I20260806 22:49:00.270941 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056275803658-18615-0/minicluster-data/ts-1/wal/instance
I20260806 22:49:00.271209 18615 external_mini_cluster.cc:1428] Setting key 6664cdab91edde54e843ef64535bc61d
I20260806 22:49:00.272260 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056275803658-18615-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056275803658-18615-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056275803658-18615-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.195:0
--local_ip_for_outbound_sockets=127.18.45.195
--webserver_interface=127.18.45.195
--webserver_port=0
--tserver_master_addrs=127.18.45.254:42581
--builtin_ntp_servers=127.18.45.212:45163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 22:49:00.368566 11743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:00.368826 11743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:00.368850 11743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:00.370146 11743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:00.370191 11743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:00.370218 11743 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.195
I20260806 22:49:00.371600 11743 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:45163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056275803658-18615-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056275803658-18615-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.45.195
--webserver_port=0
--tserver_master_addrs=127.18.45.254:42581
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11743
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.195
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056275803658-18615-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:49:00.371840 11743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:00.372151 11743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:00.377970 11752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:00.377945 11753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:00.377928 11756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:00.378468 11743 server_base.cc:1034] running on GCE node
I20260806 22:49:00.378814 11743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:00.379163 11743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:00.380311 11743 hybrid_clock.cc:648] HybridClock initialized: now 1786056540380278 us; error 53 us; skew 500 ppm
I20260806 22:49:00.381847 11743 webserver.cc:460] Webserver started at http://127.18.45.195:46489/ using document root <none> and password file <none>
I20260806 22:49:00.382201 11743 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:00.382256 11743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:00.382402 11743 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:49:00.383812 11743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056275803658-18615-0/minicluster-data/ts-2/data/instance:
uuid: "b1b3b86eaeb843e4b8fe9fbe3911836b"
format_stamp: "Formatted at 2026-08-06 22:49:00 on dist-test-slave-h3ft"
server_key: "54ed82471066f9b8637d6bedb5452ad8"
server_key_iv: "fc60e5130005d961c9ec0cd5ca752dcb"
server_key_version: "encryptionkey@0"
I20260806 22:49:00.384166 11743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056275803658-18615-0/minicluster-data/ts-2/wal/instance:
uuid: "b1b3b86eaeb843e4b8fe9fbe3911836b"
format_stamp: "Formatted at 2026-08-06 22:49:00 on dist-test-slave-h3ft"
server_key: "54ed82471066f9b8637d6bedb5452ad8"
server_key_iv: "fc60e5130005d961c9ec0cd5ca752dcb"
server_key_version: "encryptionkey@0"
I20260806 22:49:00.385627 11743 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:49:00.386487 11764 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:00.386719 11743 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:49:00.386802 11743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056275803658-18615-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056275803658-18615-0/minicluster-data/ts-2/wal
uuid: "b1b3b86eaeb843e4b8fe9fbe3911836b"
format_stamp: "Formatted at 2026-08-06 22:49:00 on dist-test-slave-h3ft"
server_key: "54ed82471066f9b8637d6bedb5452ad8"
server_key_iv: "fc60e5130005d961c9ec0cd5ca752dcb"
server_key_version: "encryptionkey@0"
I20260806 22:49:00.386862 11743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056275803658-18615-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056275803658-18615-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056275803658-18615-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:00.399080 11743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:00.399392 11743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:00.399574 11743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:00.399854 11743 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:49:00.400265 11743 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:49:00.400326 11743 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:00.400372 11743 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:49:00.400400 11743 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:00.407461 11743 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.195:36405
I20260806 22:49:00.407524 11925 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.195:36405 every 8 connection(s)
I20260806 22:49:00.407905 11743 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
I20260806 22:49:00.409731 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 11743
I20260806 22:49:00.409865 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056275803658-18615-0/minicluster-data/ts-2/wal/instance
I20260806 22:49:00.410164 18615 external_mini_cluster.cc:1428] Setting key 7ec7a86d3a4cd392495741c79f6f00f2
I20260806 22:49:00.414412 11351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:00.408820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.195:50127 (local address 127.18.45.254:42581)
0806 22:49:00.408897 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:00.408899 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:00.409171 (+ 272us) server_negotiation.cc:407] Connection header received
0806 22:49:00.409855 (+ 684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:00.409857 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:00.409874 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:00.409929 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:00.410772 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:00.411390 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:00.412505 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:00.412731 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:00.412816 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:00.413959 (+ 1143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:00.413961 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:00.413962 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:49:00.413968 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:00.413993 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:49:00.413998 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:00.414070 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:00.414211 (+ 141us) server_negotiation.cc:299] Negotiation successful
0806 22:49:00.414253 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 22:49:00.414788 11931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:00.408790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:42581 (local address 127.18.45.195:50127)
0806 22:49:00.409024 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:00.409069 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:00.409717 (+ 648us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:00.410021 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:00.410026 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:00.410035 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:00.410666 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:00.410672 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:00.411528 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:00.411529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:00.412394 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:00.412396 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:00.412823 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:00.412824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:00.412984 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:00.413709 (+ 725us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:49:00.413709 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:00.413753 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:49:00.413782 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:00.414070 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:00.414088 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:00.414092 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:49:00.414377 (+ 285us) client_negotiation.cc:240] Negotiation successful
0806 22:49:00.414516 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":114,"threads_started":1}
I20260806 22:49:00.415644 11929 heartbeater.cc:347] Connected to a master server at 127.18.45.254:42581
I20260806 22:49:00.415756 11929 heartbeater.cc:464] Registering TS with master...
I20260806 22:49:00.415944 11929 heartbeater.cc:511] Master 127.18.45.254:42581 requested a full tablet report, sending...
I20260806 22:49:00.416383 11292 ts_manager.cc:194] Registered new tserver with Master: b1b3b86eaeb843e4b8fe9fbe3911836b (127.18.45.195:36405)
I20260806 22:49:00.416778 11292 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.45.195:50127
I20260806 22:49:00.422308 18615 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:49:00.427328 11351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:00.424190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34338 (local address 127.18.45.254:42581)
0806 22:49:00.424267 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:00.424269 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:00.424379 (+ 110us) server_negotiation.cc:407] Connection header received
0806 22:49:00.424419 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:00.424420 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:00.424432 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:00.424489 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:00.424640 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:00.425180 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:00.425919 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:00.426142 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:00.426218 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:00.426996 (+ 778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:00.426998 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:00.426998 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:49:00.427070 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:00.427096 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:49:00.427101 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:00.427155 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:00.427210 (+ 55us) server_negotiation.cc:299] Negotiation successful
0806 22:49:00.427243 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 22:49:00.429338 11292 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34338:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:49:00.431454 11292 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:49:00.438704 11945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:00.434467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.195:36405 (local address 127.0.0.1:44672)
0806 22:49:00.435063 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:00.435074 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:00.435128 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:00.435470 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:00.435472 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:00.435474 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:49:00.435517 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:00.435519 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:00.436794 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:00.436795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:00.437614 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:00.437616 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:00.438377 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:00.438378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:00.438459 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:00.438474 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 22:49:00.438554 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 22:49:00.438615 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"mutex_wait_us":1,"thread_start_us":44,"threads_started":1}
I20260806 22:49:00.438717 11948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:00.434684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44672 (local address 127.18.45.195:36405)
0806 22:49:00.435285 (+ 601us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:00.435289 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:00.435299 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:49:00.435355 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:00.435357 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:00.435372 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:00.435460 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:49:00.435586 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:00.436663 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:00.437717 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:00.438266 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:00.438375 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:00.438490 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:00.438569 (+ 79us) server_negotiation.cc:299] Negotiation successful
0806 22:49:00.438612 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":541,"thread_start_us":105,"threads_started":1}
I20260806 22:49:00.439112 11939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:00.434207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.193:36637 (local address 127.0.0.1:39284)
0806 22:49:00.434415 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:00.434464 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:00.434568 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:00.435146 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:00.435147 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:00.435150 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:49:00.435219 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:00.435221 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:00.436259 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:00.436260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:00.437631 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:00.437634 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:00.438711 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:00.438712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:00.438917 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:00.438922 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:49:00.438970 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 22:49:00.439036 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":83,"threads_started":1}
I20260806 22:49:00.439286 11942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:00.434289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.194:34907 (local address 127.0.0.1:53442)
0806 22:49:00.434628 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:00.434639 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:00.434700 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:00.434978 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:00.434982 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:00.434985 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:49:00.435114 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:00.435117 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:00.436575 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:00.436576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:00.437644 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:00.437646 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:00.438610 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:00.438611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:00.438677 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:00.438682 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:49:00.438712 (+ 30us) client_negotiation.cc:240] Negotiation successful
0806 22:49:00.438758 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"mutex_wait_us":10,"thread_start_us":71,"threads_started":1}
I20260806 22:49:00.439543 11946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:00.434396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39284 (local address 127.18.45.193:36637)
0806 22:49:00.434928 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:00.434933 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:00.434946 (+ 13us) server_negotiation.cc:407] Connection header received
0806 22:49:00.435000 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:00.435019 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:00.435039 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:00.435149 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:49:00.435313 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:00.436166 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:00.437743 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:00.438509 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:00.438579 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:00.438676 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:00.439283 (+ 607us) server_negotiation.cc:299] Negotiation successful
0806 22:49:00.439439 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":115,"threads_started":1}
I20260806 22:49:00.439702 11832 tablet_service.cc:1467] Processing CreateTablet for tablet 4f67782c8e6548e9953aa123e4cb8d6d (DEFAULT_TABLE table=test-table [id=4a4575c67cdf471eb9c357ad50ba1c4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:49:00.440088 11947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:00.434433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53442 (local address 127.18.45.194:34907)
0806 22:49:00.434750 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:00.434756 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:00.434767 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:49:00.434820 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:00.434823 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:00.434851 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:00.435215 (+ 364us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:49:00.435300 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:00.436478 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:00.437742 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:00.438504 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:00.439548 (+ 1044us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:00.439913 (+ 365us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:00.439941 (+ 28us) server_negotiation.cc:299] Negotiation successful
0806 22:49:00.439991 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":133,"threads_started":1}
I20260806 22:49:00.440240 11832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f67782c8e6548e9953aa123e4cb8d6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:00.440268 11473 tablet_service.cc:1467] Processing CreateTablet for tablet 4f67782c8e6548e9953aa123e4cb8d6d (DEFAULT_TABLE table=test-table [id=4a4575c67cdf471eb9c357ad50ba1c4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:49:00.440694 11473 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f67782c8e6548e9953aa123e4cb8d6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:00.440785 11658 tablet_service.cc:1467] Processing CreateTablet for tablet 4f67782c8e6548e9953aa123e4cb8d6d (DEFAULT_TABLE table=test-table [id=4a4575c67cdf471eb9c357ad50ba1c4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:49:00.441202 11658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f67782c8e6548e9953aa123e4cb8d6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:00.443979 11949 tablet_bootstrap.cc:492] T 4f67782c8e6548e9953aa123e4cb8d6d P a7bf7e05910e4350bfab1ba0d220d85e: Bootstrap starting.
I20260806 22:49:00.444465 11950 tablet_bootstrap.cc:492] T 4f67782c8e6548e9953aa123e4cb8d6d P b1b3b86eaeb843e4b8fe9fbe3911836b: Bootstrap starting.
I20260806 22:49:00.444932 11949 tablet_bootstrap.cc:654] T 4f67782c8e6548e9953aa123e4cb8d6d P a7bf7e05910e4350bfab1ba0d220d85e: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:00.445264 11950 tablet_bootstrap.cc:654] T 4f67782c8e6548e9953aa123e4cb8d6d P b1b3b86eaeb843e4b8fe9fbe3911836b: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:00.445552 11949 log.cc:824] T 4f67782c8e6548e9953aa123e4cb8d6d P a7bf7e05910e4350bfab1ba0d220d85e: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:00.445608 11950 log.cc:824] T 4f67782c8e6548e9953aa123e4cb8d6d P b1b3b86eaeb843e4b8fe9fbe3911836b: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:00.446017 11951 tablet_bootstrap.cc:492] T 4f67782c8e6548e9953aa123e4cb8d6d P e6e3e92928664ddcb07260a9e023ed9c: Bootstrap starting.
I20260806 22:49:00.446444 11949 tablet_bootstrap.cc:492] T 4f67782c8e6548e9953aa123e4cb8d6d P a7bf7e05910e4350bfab1ba0d220d85e: No bootstrap required, opened a new log
I20260806 22:49:00.446481 11950 tablet_bootstrap.cc:492] T 4f67782c8e6548e9953aa123e4cb8d6d P b1b3b86eaeb843e4b8fe9fbe3911836b: No bootstrap required, opened a new log
I20260806 22:49:00.446543 11949 ts_tablet_manager.cc:1397] T 4f67782c8e6548e9953aa123e4cb8d6d P a7bf7e05910e4350bfab1ba0d220d85e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:49:00.446556 11950 ts_tablet_manager.cc:1397] T 4f67782c8e6548e9953aa123e4cb8d6d P b1b3b86eaeb843e4b8fe9fbe3911836b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:49:00.447830 11951 tablet_bootstrap.cc:654] T 4f67782c8e6548e9953aa123e4cb8d6d P e6e3e92928664ddcb07260a9e023ed9c: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:00.448220 11951 log.cc:824] T 4f67782c8e6548e9953aa123e4cb8d6d P e6e3e92928664ddcb07260a9e023ed9c: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:00.448963 11950 raft_consensus.cc:348] T 4f67782c8e6548e9953aa123e4cb8d6d P b1b3b86eaeb843e4b8fe9fbe3911836b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7bf7e05910e4350bfab1ba0d220d85e" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 36637 } } peers { permanent_uuid: "e6e3e92928664ddcb07260a9e023ed9c" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 34907 } } peers { permanent_uuid: "b1b3b86eaeb843e4b8fe9fbe3911836b" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 36405 } }
I20260806 22:49:00.449124 11951 tablet_bootstrap.cc:492] T 4f67782c8e6548e9953aa123e4cb8d6d P e6e3e92928664ddcb07260a9e023ed9c: No bootstrap required, opened a new log
I20260806 22:49:00.449162 11950 raft_consensus.cc:374] T 4f67782c8e6548e9953aa123e4cb8d6d P b1b3b86eaeb843e4b8fe9fbe3911836b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:00.449210 11950 raft_consensus.cc:729] T 4f67782c8e6548e9953aa123e4cb8d6d P b1b3b86eaeb843e4b8fe9fbe3911836b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1b3b86eaeb843e4b8fe9fbe3911836b, State: Initialized, Role: FOLLOWER
I20260806 22:49:00.449275 11951 ts_tablet_manager.cc:1397] T 4f67782c8e6548e9953aa123e4cb8d6d P e6e3e92928664ddcb07260a9e023ed9c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:49:00.449371 11950 consensus_queue.cc:260] T 4f67782c8e6548e9953aa123e4cb8d6d P b1b3b86eaeb843e4b8fe9fbe3911836b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7bf7e05910e4350bfab1ba0d220d85e" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 36637 } } peers { permanent_uuid: "e6e3e92928664ddcb07260a9e023ed9c" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 34907 } } peers { permanent_uuid: "b1b3b86eaeb843e4b8fe9fbe3911836b" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 36405 } }
I20260806 22:49:00.449467 11949 raft_consensus.cc:348] T 4f67782c8e6548e9953aa123e4cb8d6d P a7bf7e05910e4350bfab1ba0d220d85e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7bf7e05910e4350bfab1ba0d220d85e" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 36637 } } peers { permanent_uuid: "e6e3e92928664ddcb07260a9e023ed9c" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 34907 } } peers { permanent_uuid: "b1b3b86eaeb843e4b8fe9fbe3911836b" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 36405 } }
I20260806 22:49:00.449663 11949 raft_consensus.cc:374] T 4f67782c8e6548e9953aa123e4cb8d6d P a7bf7e05910e4350bfab1ba0d220d85e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:00.449689 11950 ts_tablet_manager.cc:1428] T 4f67782c8e6548e9953aa123e4cb8d6d P b1b3b86eaeb843e4b8fe9fbe3911836b: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 22:49:00.449712 11949 raft_consensus.cc:729] T 4f67782c8e6548e9953aa123e4cb8d6d P a7bf7e05910e4350bfab1ba0d220d85e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7bf7e05910e4350bfab1ba0d220d85e, State: Initialized, Role: FOLLOWER
I20260806 22:49:00.449824 11929 heartbeater.cc:503] Master 127.18.45.254:42581 was elected leader, sending a full tablet report...
I20260806 22:49:00.449873 11949 consensus_queue.cc:260] T 4f67782c8e6548e9953aa123e4cb8d6d P a7bf7e05910e4350bfab1ba0d220d85e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7bf7e05910e4350bfab1ba0d220d85e" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 36637 } } peers { permanent_uuid: "e6e3e92928664ddcb07260a9e023ed9c" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 34907 } } peers { permanent_uuid: "b1b3b86eaeb843e4b8fe9fbe3911836b" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 36405 } }
I20260806 22:49:00.450155 11949 ts_tablet_manager.cc:1428] T 4f67782c8e6548e9953aa123e4cb8d6d P a7bf7e05910e4350bfab1ba0d220d85e: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260806 22:49:00.450268 11571 heartbeater.cc:503] Master 127.18.45.254:42581 was elected leader, sending a full tablet report...
I20260806 22:49:00.451956 11951 raft_consensus.cc:348] T 4f67782c8e6548e9953aa123e4cb8d6d P e6e3e92928664ddcb07260a9e023ed9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7bf7e05910e4350bfab1ba0d220d85e" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 36637 } } peers { permanent_uuid: "e6e3e92928664ddcb07260a9e023ed9c" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 34907 } } peers { permanent_uuid: "b1b3b86eaeb843e4b8fe9fbe3911836b" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 36405 } }
I20260806 22:49:00.452082 11951 raft_consensus.cc:374] T 4f67782c8e6548e9953aa123e4cb8d6d P e6e3e92928664ddcb07260a9e023ed9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:00.452103 11951 raft_consensus.cc:729] T 4f67782c8e6548e9953aa123e4cb8d6d P e6e3e92928664ddcb07260a9e023ed9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6e3e92928664ddcb07260a9e023ed9c, State: Initialized, Role: FOLLOWER
I20260806 22:49:00.452215 11951 consensus_queue.cc:260] T 4f67782c8e6548e9953aa123e4cb8d6d P e6e3e92928664ddcb07260a9e023ed9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7bf7e05910e4350bfab1ba0d220d85e" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 36637 } } peers { permanent_uuid: "e6e3e92928664ddcb07260a9e023ed9c" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 34907 } } peers { permanent_uuid: "b1b3b86eaeb843e4b8fe9fbe3911836b" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 36405 } }
I20260806 22:49:00.452508 11951 ts_tablet_manager.cc:1428] T 4f67782c8e6548e9953aa123e4cb8d6d P e6e3e92928664ddcb07260a9e023ed9c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:49:00.452582 11739 heartbeater.cc:503] Master 127.18.45.254:42581 was elected leader, sending a full tablet report...
W20260806 22:49:00.511401 11740 tablet.cc:2369] T 4f67782c8e6548e9953aa123e4cb8d6d P e6e3e92928664ddcb07260a9e023ed9c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:49:00.613375 11572 tablet.cc:2369] T 4f67782c8e6548e9953aa123e4cb8d6d P a7bf7e05910e4350bfab1ba0d220d85e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:49:00.627750 11956 raft_consensus.cc:482] T 4f67782c8e6548e9953aa123e4cb8d6d P a7bf7e05910e4350bfab1ba0d220d85e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:49:00.627930 11956 raft_consensus.cc:504] T 4f67782c8e6548e9953aa123e4cb8d6d P a7bf7e05910e4350bfab1ba0d220d85e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7bf7e05910e4350bfab1ba0d220d85e" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 36637 } } peers { permanent_uuid: "e6e3e92928664ddcb07260a9e023ed9c" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 34907 } } peers { permanent_uuid: "b1b3b86eaeb843e4b8fe9fbe3911836b" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 36405 } }
I20260806 22:49:00.628376 11956 leader_election.cc:288] T 4f67782c8e6548e9953aa123e4cb8d6d P a7bf7e05910e4350bfab1ba0d220d85e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e6e3e92928664ddcb07260a9e023ed9c (127.18.45.194:34907), b1b3b86eaeb843e4b8fe9fbe3911836b (127.18.45.195:36405)
I20260806 22:49:00.631772 11948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:00.629049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:40291 (local address 127.18.45.195:36405)
0806 22:49:00.629147 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:00.629149 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:00.629159 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:49:00.629199 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:00.629200 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:00.629213 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:00.629279 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:49:00.629459 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:00.629939 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:00.630922 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:00.631393 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:00.631481 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:00.631492 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:00.631640 (+ 148us) server_negotiation.cc:299] Negotiation successful
0806 22:49:00.631684 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 22:49:00.631793 11947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:00.628741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:41751 (local address 127.18.45.194:34907)
0806 22:49:00.628811 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:00.628813 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:00.628945 (+ 132us) server_negotiation.cc:407] Connection header received
0806 22:49:00.629052 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:00.629053 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:00.629071 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:00.629140 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:49:00.629344 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:00.629877 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:00.630938 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:00.631404 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:00.631498 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:00.631509 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:00.631670 (+ 161us) server_negotiation.cc:299] Negotiation successful
0806 22:49:00.631715 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 22:49:00.631805 11958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:00.628630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.194:34907 (local address 127.18.45.193:41751)
0806 22:49:00.628852 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:00.628866 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:00.628953 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:00.629153 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:00.629155 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:00.629157 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:49:00.629258 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:00.629261 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:00.629970 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:00.629971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:00.630828 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:00.630830 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:00.631510 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:00.631511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:00.631580 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:00.631584 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:49:00.631660 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 22:49:00.631702 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"mutex_wait_us":11,"thread_start_us":127,"threads_started":1}
I20260806 22:49:00.631798 11959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:00.628895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.195:36405 (local address 127.18.45.193:40291)
0806 22:49:00.629056 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:00.629064 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:00.629131 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:00.629283 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:00.629284 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:00.629286 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:49:00.629382 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:00.629384 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:00.630028 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:00.630030 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:00.630822 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:00.630824 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:00.631482 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:00.631483 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:00.631562 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:00.631566 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:49:00.631632 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 22:49:00.631676 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":96,"threads_started":1}
I20260806 22:49:00.632320 11684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f67782c8e6548e9953aa123e4cb8d6d" candidate_uuid: "a7bf7e05910e4350bfab1ba0d220d85e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6e3e92928664ddcb07260a9e023ed9c" is_pre_election: true
I20260806 22:49:00.632551 11684 raft_consensus.cc:2454] T 4f67782c8e6548e9953aa123e4cb8d6d P e6e3e92928664ddcb07260a9e023ed9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a7bf7e05910e4350bfab1ba0d220d85e in term 0.
I20260806 22:49:00.632582 11856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f67782c8e6548e9953aa123e4cb8d6d" candidate_uuid: "a7bf7e05910e4350bfab1ba0d220d85e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1b3b86eaeb843e4b8fe9fbe3911836b" is_pre_election: true
I20260806 22:49:00.632793 11856 raft_consensus.cc:2454] T 4f67782c8e6548e9953aa123e4cb8d6d P b1b3b86eaeb843e4b8fe9fbe3911836b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a7bf7e05910e4350bfab1ba0d220d85e in term 0.
I20260806 22:49:00.632885 11420 leader_election.cc:302] T 4f67782c8e6548e9953aa123e4cb8d6d P a7bf7e05910e4350bfab1ba0d220d85e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a7bf7e05910e4350bfab1ba0d220d85e, e6e3e92928664ddcb07260a9e023ed9c; no voters:
I20260806 22:49:00.633057 11956 raft_consensus.cc:2781] T 4f67782c8e6548e9953aa123e4cb8d6d P a7bf7e05910e4350bfab1ba0d220d85e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:49:00.633124 11956 raft_consensus.cc:482] T 4f67782c8e6548e9953aa123e4cb8d6d P a7bf7e05910e4350bfab1ba0d220d85e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:49:00.633153 11956 raft_consensus.cc:3037] T 4f67782c8e6548e9953aa123e4cb8d6d P a7bf7e05910e4350bfab1ba0d220d85e [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:00.634088 11956 raft_consensus.cc:504] T 4f67782c8e6548e9953aa123e4cb8d6d P a7bf7e05910e4350bfab1ba0d220d85e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7bf7e05910e4350bfab1ba0d220d85e" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 36637 } } peers { permanent_uuid: "e6e3e92928664ddcb07260a9e023ed9c" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 34907 } } peers { permanent_uuid: "b1b3b86eaeb843e4b8fe9fbe3911836b" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 36405 } }
I20260806 22:49:00.634249 11956 leader_election.cc:288] T 4f67782c8e6548e9953aa123e4cb8d6d P a7bf7e05910e4350bfab1ba0d220d85e [CANDIDATE]: Term 1 election: Requested vote from peers e6e3e92928664ddcb07260a9e023ed9c (127.18.45.194:34907), b1b3b86eaeb843e4b8fe9fbe3911836b (127.18.45.195:36405)
I20260806 22:49:00.634475 11684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f67782c8e6548e9953aa123e4cb8d6d" candidate_uuid: "a7bf7e05910e4350bfab1ba0d220d85e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6e3e92928664ddcb07260a9e023ed9c"
I20260806 22:49:00.634584 11684 raft_consensus.cc:3037] T 4f67782c8e6548e9953aa123e4cb8d6d P e6e3e92928664ddcb07260a9e023ed9c [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:00.635591 11684 raft_consensus.cc:2454] T 4f67782c8e6548e9953aa123e4cb8d6d P e6e3e92928664ddcb07260a9e023ed9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a7bf7e05910e4350bfab1ba0d220d85e in term 1.
I20260806 22:49:00.635730 11856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f67782c8e6548e9953aa123e4cb8d6d" candidate_uuid: "a7bf7e05910e4350bfab1ba0d220d85e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1b3b86eaeb843e4b8fe9fbe3911836b"
I20260806 22:49:00.635788 11420 leader_election.cc:302] T 4f67782c8e6548e9953aa123e4cb8d6d P a7bf7e05910e4350bfab1ba0d220d85e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a7bf7e05910e4350bfab1ba0d220d85e, e6e3e92928664ddcb07260a9e023ed9c; no voters:
I20260806 22:49:00.635885 11956 raft_consensus.cc:2781] T 4f67782c8e6548e9953aa123e4cb8d6d P a7bf7e05910e4350bfab1ba0d220d85e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:49:00.635829 11856 raft_consensus.cc:3037] T 4f67782c8e6548e9953aa123e4cb8d6d P b1b3b86eaeb843e4b8fe9fbe3911836b [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:00.636127 11956 raft_consensus.cc:686] T 4f67782c8e6548e9953aa123e4cb8d6d P a7bf7e05910e4350bfab1ba0d220d85e [term 1 LEADER]: Becoming Leader. State: Replica: a7bf7e05910e4350bfab1ba0d220d85e, State: Running, Role: LEADER
I20260806 22:49:00.636270 11956 consensus_queue.cc:237] T 4f67782c8e6548e9953aa123e4cb8d6d P a7bf7e05910e4350bfab1ba0d220d85e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7bf7e05910e4350bfab1ba0d220d85e" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 36637 } } peers { permanent_uuid: "e6e3e92928664ddcb07260a9e023ed9c" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 34907 } } peers { permanent_uuid: "b1b3b86eaeb843e4b8fe9fbe3911836b" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 36405 } }
I20260806 22:49:00.636981 11856 raft_consensus.cc:2454] T 4f67782c8e6548e9953aa123e4cb8d6d P b1b3b86eaeb843e4b8fe9fbe3911836b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a7bf7e05910e4350bfab1ba0d220d85e in term 1.
I20260806 22:49:00.637713 11292 catalog_manager.cc:5495] T 4f67782c8e6548e9953aa123e4cb8d6d P a7bf7e05910e4350bfab1ba0d220d85e reported cstate change: term changed from 0 to 1, leader changed from <none> to a7bf7e05910e4350bfab1ba0d220d85e (127.18.45.193). New cstate: current_term: 1 leader_uuid: "a7bf7e05910e4350bfab1ba0d220d85e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7bf7e05910e4350bfab1ba0d220d85e" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 36637 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e6e3e92928664ddcb07260a9e023ed9c" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 34907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1b3b86eaeb843e4b8fe9fbe3911836b" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 36405 } health_report { overall_health: UNKNOWN } } }
W20260806 22:49:00.658629 11930 tablet.cc:2369] T 4f67782c8e6548e9953aa123e4cb8d6d P b1b3b86eaeb843e4b8fe9fbe3911836b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:49:00.669163 11946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:00.666580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39300 (local address 127.18.45.193:36637)
0806 22:49:00.666657 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:00.666659 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:00.666688 (+ 29us) server_negotiation.cc:407] Connection header received
0806 22:49:00.666738 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:00.666739 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:00.666769 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:00.666828 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:49:00.666984 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:00.667572 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:00.668364 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:00.668559 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:00.668613 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:00.668768 (+ 155us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:49:00.668896 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:49:00.668973 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:00.669007 (+ 34us) server_negotiation.cc:299] Negotiation successful
0806 22:49:00.669028 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 22:49:00.671686 11856 raft_consensus.cc:1261] T 4f67782c8e6548e9953aa123e4cb8d6d P b1b3b86eaeb843e4b8fe9fbe3911836b [term 1 FOLLOWER]: Refusing update from remote peer a7bf7e05910e4350bfab1ba0d220d85e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:49:00.671686 11684 raft_consensus.cc:1261] T 4f67782c8e6548e9953aa123e4cb8d6d P e6e3e92928664ddcb07260a9e023ed9c [term 1 FOLLOWER]: Refusing update from remote peer a7bf7e05910e4350bfab1ba0d220d85e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:49:00.672154 11956 consensus_queue.cc:1035] T 4f67782c8e6548e9953aa123e4cb8d6d P a7bf7e05910e4350bfab1ba0d220d85e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6e3e92928664ddcb07260a9e023ed9c" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 34907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:49:00.672338 11960 consensus_queue.cc:1035] T 4f67782c8e6548e9953aa123e4cb8d6d P a7bf7e05910e4350bfab1ba0d220d85e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1b3b86eaeb843e4b8fe9fbe3911836b" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 36405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:49:00.675817 11965 mvcc.cc:205] Tried to move back new op lower bound from 7315687590587478016 to 7315687590446981120. Current Snapshot: MvccSnapshot[applied={T|T < 7315687590587478016}]
I20260806 22:49:00.676046 11966 mvcc.cc:205] Tried to move back new op lower bound from 7315687590587478016 to 7315687590446981120. Current Snapshot: MvccSnapshot[applied={T|T < 7315687590587478016}]
I20260806 22:49:00.678486 11292 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34338:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:49:00.678613 11292 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34338:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:49:00.679739 11292 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 003fbbf432c64e139279c18ba531a6f9: Sending DeleteTablet for 3 replicas of tablet 4f67782c8e6548e9953aa123e4cb8d6d
I20260806 22:49:00.680208 11473 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f67782c8e6548e9953aa123e4cb8d6d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:49:00 UTC) from {username='slave'} at 127.0.0.1:39284
I20260806 22:49:00.680239 11658 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f67782c8e6548e9953aa123e4cb8d6d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:49:00 UTC) from {username='slave'} at 127.0.0.1:53442
I20260806 22:49:00.680233 11832 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f67782c8e6548e9953aa123e4cb8d6d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:49:00 UTC) from {username='slave'} at 127.0.0.1:44672
I20260806 22:49:00.680660 11975 tablet_replica.cc:330] T 4f67782c8e6548e9953aa123e4cb8d6d P e6e3e92928664ddcb07260a9e023ed9c: stopping tablet replica
I20260806 22:49:00.680796 11975 raft_consensus.cc:2229] T 4f67782c8e6548e9953aa123e4cb8d6d P e6e3e92928664ddcb07260a9e023ed9c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:49:00.680903 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 11384
I20260806 22:49:00.680948 11975 raft_consensus.cc:2258] T 4f67782c8e6548e9953aa123e4cb8d6d P e6e3e92928664ddcb07260a9e023ed9c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:49:00.691605 11976 tablet_replica.cc:330] T 4f67782c8e6548e9953aa123e4cb8d6d P b1b3b86eaeb843e4b8fe9fbe3911836b: stopping tablet replica
I20260806 22:49:00.691833 11976 raft_consensus.cc:2229] T 4f67782c8e6548e9953aa123e4cb8d6d P b1b3b86eaeb843e4b8fe9fbe3911836b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:49:00.692049 11976 raft_consensus.cc:2258] T 4f67782c8e6548e9953aa123e4cb8d6d P b1b3b86eaeb843e4b8fe9fbe3911836b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:49:00.692730 11975 ts_tablet_manager.cc:1905] T 4f67782c8e6548e9953aa123e4cb8d6d P e6e3e92928664ddcb07260a9e023ed9c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:49:00.695259 11975 ts_tablet_manager.cc:1918] T 4f67782c8e6548e9953aa123e4cb8d6d P e6e3e92928664ddcb07260a9e023ed9c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:49:00.695369 11975 log.cc:1196] T 4f67782c8e6548e9953aa123e4cb8d6d P e6e3e92928664ddcb07260a9e023ed9c: Deleting WAL directory at /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056275803658-18615-0/minicluster-data/ts-1/wal/wals/4f67782c8e6548e9953aa123e4cb8d6d
I20260806 22:49:00.695519 11976 ts_tablet_manager.cc:1905] T 4f67782c8e6548e9953aa123e4cb8d6d P b1b3b86eaeb843e4b8fe9fbe3911836b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:49:00.695703 11975 ts_tablet_manager.cc:1939] T 4f67782c8e6548e9953aa123e4cb8d6d P e6e3e92928664ddcb07260a9e023ed9c: Deleting consensus metadata
I20260806 22:49:00.696346 11275 catalog_manager.cc:4841] TS e6e3e92928664ddcb07260a9e023ed9c (127.18.45.194:34907): tablet 4f67782c8e6548e9953aa123e4cb8d6d (table test-table [id=4a4575c67cdf471eb9c357ad50ba1c4a]) successfully deleted
I20260806 22:49:00.697415 11976 ts_tablet_manager.cc:1918] T 4f67782c8e6548e9953aa123e4cb8d6d P b1b3b86eaeb843e4b8fe9fbe3911836b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:49:00.697492 11976 log.cc:1196] T 4f67782c8e6548e9953aa123e4cb8d6d P b1b3b86eaeb843e4b8fe9fbe3911836b: Deleting WAL directory at /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786056275803658-18615-0/minicluster-data/ts-2/wal/wals/4f67782c8e6548e9953aa123e4cb8d6d
I20260806 22:49:00.697801 11976 ts_tablet_manager.cc:1939] T 4f67782c8e6548e9953aa123e4cb8d6d P b1b3b86eaeb843e4b8fe9fbe3911836b: Deleting consensus metadata
I20260806 22:49:00.698315 11275 catalog_manager.cc:4841] TS b1b3b86eaeb843e4b8fe9fbe3911836b (127.18.45.195:36405): tablet 4f67782c8e6548e9953aa123e4cb8d6d (table test-table [id=4a4575c67cdf471eb9c357ad50ba1c4a]) successfully deleted
W20260806 22:49:00.698781 11273 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.45.193:36637 (error 108)
I20260806 22:49:00.699113 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 11574
I20260806 22:49:00.699404 11942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:00.699248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.193:36637 (local address 127.0.0.1:39304)
0806 22:49:00.699309 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:00.699353 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.193:36637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260806 22:49:00.703264 11273 catalog_manager.cc:4584] TS a7bf7e05910e4350bfab1ba0d220d85e (127.18.45.193:36637): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4f67782c8e6548e9953aa123e4cb8d6d: Network error: Client connection negotiation failed: client connection to 127.18.45.193:36637: connect: Connection refused (error 111)
I20260806 22:49:00.719539 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 11743
I20260806 22:49:00.731114 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 11249
2026-08-06T22:49:00Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1004 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:49:00 dist-test-slave-h3ft krb5kdc[11981](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:49:00 dist-test-slave-h3ft krb5kdc[11981](info): set up 2 sockets
Aug 06 22:49:00 dist-test-slave-h3ft krb5kdc[11981](info): commencing operation
krb5kdc: starting...
W20260806 22:49:02.005196 18615 mini_kdc.cc:121] Time spent starting KDC: real 1.248s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Aug 06 22:49:02 dist-test-slave-h3ft krb5kdc[11981](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056542, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:49:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:49:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.45.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab.
Entry for principal kudu/127.18.45.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab.
I20260806 22:49:02.163239 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:45249
--webserver_interface=127.18.45.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:46339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.45.254:45249
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:49:02.260172 11998 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:02.260461 11998 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:02.260495 11998 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:02.261884 11998 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:49:02.261943 11998 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:02.261960 11998 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:02.261972 11998 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:49:02.261984 11998 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:49:02.263823 11998 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:46339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.45.254:45249
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:45249
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.45.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11998
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:49:02.264102 11998 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:02.264336 11998 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:02.270581 12007 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:02.270571 12008 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:02.270637 11998 server_base.cc:1034] running on GCE node
W20260806 22:49:02.270578 12012 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:02.271217 11998 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:02.271626 11998 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:02.272796 11998 hybrid_clock.cc:648] HybridClock initialized: now 1786056542272762 us; error 62 us; skew 500 ppm
Aug 06 22:49:02 dist-test-slave-h3ft krb5kdc[11981](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056542, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:02.275969 11998 init.cc:377] Logged in from keytab as kudu/127.18.45.254@KRBTEST.COM (short username kudu)
I20260806 22:49:02.276566 11998 webserver.cc:460] Webserver started at http://127.18.45.254:46305/ using document root <none> and password file <none>
I20260806 22:49:02.276836 11998 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:02.276909 11998 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:02.277096 11998 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:49:02.278436 11998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/minicluster-data/master-0/data/instance:
uuid: "504045cfcea74b588d76ad9aaa8f6cc5"
format_stamp: "Formatted at 2026-08-06 22:49:02 on dist-test-slave-h3ft"
server_key: "daa2bc37406cc12490d588d1d15a21b0"
server_key_iv: "ce00f11387cd65453e754a993bc248b2"
server_key_version: "encryptionkey@0"
I20260806 22:49:02.278870 11998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/minicluster-data/master-0/wal/instance:
uuid: "504045cfcea74b588d76ad9aaa8f6cc5"
format_stamp: "Formatted at 2026-08-06 22:49:02 on dist-test-slave-h3ft"
server_key: "daa2bc37406cc12490d588d1d15a21b0"
server_key_iv: "ce00f11387cd65453e754a993bc248b2"
server_key_version: "encryptionkey@0"
I20260806 22:49:02.280697 11998 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260806 22:49:02.281643 12021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:02.281848 11998 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:49:02.281980 11998 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/minicluster-data/master-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/minicluster-data/master-0/wal
uuid: "504045cfcea74b588d76ad9aaa8f6cc5"
format_stamp: "Formatted at 2026-08-06 22:49:02 on dist-test-slave-h3ft"
server_key: "daa2bc37406cc12490d588d1d15a21b0"
server_key_iv: "ce00f11387cd65453e754a993bc248b2"
server_key_version: "encryptionkey@0"
I20260806 22:49:02.282074 11998 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:02.295655 11998 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:02.298358 11998 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:02.298557 11998 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:02.303957 11998 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.254:45249
I20260806 22:49:02.304003 12087 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.254:45249 every 8 connection(s)
I20260806 22:49:02.304450 11998 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
I20260806 22:49:02.305652 12088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:02.308876 12088 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 504045cfcea74b588d76ad9aaa8f6cc5: Bootstrap starting.
I20260806 22:49:02.309633 12088 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 504045cfcea74b588d76ad9aaa8f6cc5: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:02.309973 12088 log.cc:824] T 00000000000000000000000000000000 P 504045cfcea74b588d76ad9aaa8f6cc5: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:02.311069 12088 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 504045cfcea74b588d76ad9aaa8f6cc5: No bootstrap required, opened a new log
I20260806 22:49:02.311077 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 11998
I20260806 22:49:02.311234 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/minicluster-data/master-0/wal/instance
I20260806 22:49:02.311524 18615 external_mini_cluster.cc:1428] Setting key f088961d6a46eb0ebaffa2fbfb700b9a
I20260806 22:49:02.313688 12088 raft_consensus.cc:348] T 00000000000000000000000000000000 P 504045cfcea74b588d76ad9aaa8f6cc5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "504045cfcea74b588d76ad9aaa8f6cc5" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 45249 } }
I20260806 22:49:02.313912 12088 raft_consensus.cc:374] T 00000000000000000000000000000000 P 504045cfcea74b588d76ad9aaa8f6cc5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:02.313962 12088 raft_consensus.cc:729] T 00000000000000000000000000000000 P 504045cfcea74b588d76ad9aaa8f6cc5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 504045cfcea74b588d76ad9aaa8f6cc5, State: Initialized, Role: FOLLOWER
I20260806 22:49:02.314184 12088 consensus_queue.cc:260] T 00000000000000000000000000000000 P 504045cfcea74b588d76ad9aaa8f6cc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "504045cfcea74b588d76ad9aaa8f6cc5" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 45249 } }
I20260806 22:49:02.314302 12088 raft_consensus.cc:388] T 00000000000000000000000000000000 P 504045cfcea74b588d76ad9aaa8f6cc5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:49:02.314347 12088 raft_consensus.cc:482] T 00000000000000000000000000000000 P 504045cfcea74b588d76ad9aaa8f6cc5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:49:02.314410 12088 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 504045cfcea74b588d76ad9aaa8f6cc5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:02.315790 12088 raft_consensus.cc:504] T 00000000000000000000000000000000 P 504045cfcea74b588d76ad9aaa8f6cc5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "504045cfcea74b588d76ad9aaa8f6cc5" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 45249 } }
I20260806 22:49:02.315919 12088 leader_election.cc:302] T 00000000000000000000000000000000 P 504045cfcea74b588d76ad9aaa8f6cc5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 504045cfcea74b588d76ad9aaa8f6cc5; no voters:
I20260806 22:49:02.316164 12088 leader_election.cc:288] T 00000000000000000000000000000000 P 504045cfcea74b588d76ad9aaa8f6cc5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:49:02.316217 12095 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 504045cfcea74b588d76ad9aaa8f6cc5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:49:02.316489 12095 raft_consensus.cc:686] T 00000000000000000000000000000000 P 504045cfcea74b588d76ad9aaa8f6cc5 [term 1 LEADER]: Becoming Leader. State: Replica: 504045cfcea74b588d76ad9aaa8f6cc5, State: Running, Role: LEADER
I20260806 22:49:02.316627 12088 sys_catalog.cc:564] T 00000000000000000000000000000000 P 504045cfcea74b588d76ad9aaa8f6cc5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:49:02.316663 12095 consensus_queue.cc:237] T 00000000000000000000000000000000 P 504045cfcea74b588d76ad9aaa8f6cc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "504045cfcea74b588d76ad9aaa8f6cc5" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 45249 } }
I20260806 22:49:02.317117 12097 sys_catalog.cc:455] T 00000000000000000000000000000000 P 504045cfcea74b588d76ad9aaa8f6cc5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "504045cfcea74b588d76ad9aaa8f6cc5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "504045cfcea74b588d76ad9aaa8f6cc5" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 45249 } } }
I20260806 22:49:02.317106 12095 sys_catalog.cc:455] T 00000000000000000000000000000000 P 504045cfcea74b588d76ad9aaa8f6cc5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 504045cfcea74b588d76ad9aaa8f6cc5. Latest consensus state: current_term: 1 leader_uuid: "504045cfcea74b588d76ad9aaa8f6cc5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "504045cfcea74b588d76ad9aaa8f6cc5" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 45249 } } }
I20260806 22:49:02.317292 12095 sys_catalog.cc:458] T 00000000000000000000000000000000 P 504045cfcea74b588d76ad9aaa8f6cc5 [sys.catalog]: This master's current role is: LEADER
I20260806 22:49:02.317281 12097 sys_catalog.cc:458] T 00000000000000000000000000000000 P 504045cfcea74b588d76ad9aaa8f6cc5 [sys.catalog]: This master's current role is: LEADER
I20260806 22:49:02.317570 12106 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:49:02.318207 12106 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:49:02.320463 12106 catalog_manager.cc:1347] Generated new cluster ID: 94886cf492034da28c98a87a2171c8bc
I20260806 22:49:02.320526 12106 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:49:02 dist-test-slave-h3ft krb5kdc[11981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056542, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:49:02.329881 12094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:02.312260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34186 (local address 127.18.45.254:45249)
0806 22:49:02.312453 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:02.312463 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:02.312499 (+ 36us) server_negotiation.cc:407] Connection header received
0806 22:49:02.313673 (+ 1174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:02.313684 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:02.313752 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:02.313898 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:02.314962 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:02.316131 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:02.317469 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:02.318289 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:02.318407 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:02.322059 (+ 3652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:02.322064 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:02.322065 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:02.322081 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:02.327425 (+ 5344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:02.327958 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:02.327962 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:02.327963 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:02.328172 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:02.328431 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:02.328432 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:02.328433 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:02.328913 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:02.329204 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:02.329350 (+ 146us) server_negotiation.cc:299] Negotiation successful
0806 22:49:02.329575 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":95,"threads_started":1}
I20260806 22:49:02.332854 12106 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:49:02.333489 12106 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:49:02.336790 12106 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 504045cfcea74b588d76ad9aaa8f6cc5: Generated new TSK 0
I20260806 22:49:02.337046 12106 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.45.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab.
Entry for principal kudu/127.18.45.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab.
I20260806 22:49:02.385470 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.193:0
--local_ip_for_outbound_sockets=127.18.45.193
--webserver_interface=127.18.45.193
--webserver_port=0
--tserver_master_addrs=127.18.45.254:45249
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab
--principal=kudu/127.18.45.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:46339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:49:02.522701 12125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:02.522982 12125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:02.523043 12125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:02.524277 12125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:02.524355 12125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:02.524421 12125 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.193
I20260806 22:49:02.526321 12125 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:46339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab
--principal=kudu/127.18.45.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.45.193
--webserver_port=0
--tserver_master_addrs=127.18.45.254:45249
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12125
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.193
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:49:02.526644 12125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:02.528010 12125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:02.534906 12134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:02.535171 12125 server_base.cc:1034] running on GCE node
W20260806 22:49:02.534946 12136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:02.534955 12131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:02.535667 12125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:02.535988 12125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:02.537163 12125 hybrid_clock.cc:648] HybridClock initialized: now 1786056542537140 us; error 41 us; skew 500 ppm
Aug 06 22:49:02 dist-test-slave-h3ft krb5kdc[11981](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056542, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:02.545171 12125 init.cc:377] Logged in from keytab as kudu/127.18.45.193@KRBTEST.COM (short username kudu)
I20260806 22:49:02.545776 12125 webserver.cc:460] Webserver started at http://127.18.45.193:46653/ using document root <none> and password file <none>
I20260806 22:49:02.546052 12125 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:02.546098 12125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:02.546211 12125 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:49:02.547557 12125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/minicluster-data/ts-0/data/instance:
uuid: "d6c0ae92ca814f0aa78053903d715a15"
format_stamp: "Formatted at 2026-08-06 22:49:02 on dist-test-slave-h3ft"
server_key: "358382a1ef69b56236afe7b3f6a6bd5f"
server_key_iv: "f05236aa24b983928d1504ed1b222b40"
server_key_version: "encryptionkey@0"
I20260806 22:49:02.547911 12125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/minicluster-data/ts-0/wal/instance:
uuid: "d6c0ae92ca814f0aa78053903d715a15"
format_stamp: "Formatted at 2026-08-06 22:49:02 on dist-test-slave-h3ft"
server_key: "358382a1ef69b56236afe7b3f6a6bd5f"
server_key_iv: "f05236aa24b983928d1504ed1b222b40"
server_key_version: "encryptionkey@0"
I20260806 22:49:02.549417 12125 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260806 22:49:02.550454 12144 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:02.550697 12125 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:49:02.550819 12125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/minicluster-data/ts-0/wal
uuid: "d6c0ae92ca814f0aa78053903d715a15"
format_stamp: "Formatted at 2026-08-06 22:49:02 on dist-test-slave-h3ft"
server_key: "358382a1ef69b56236afe7b3f6a6bd5f"
server_key_iv: "f05236aa24b983928d1504ed1b222b40"
server_key_version: "encryptionkey@0"
I20260806 22:49:02.550909 12125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:02.559067 12125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:02.561714 12125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:02.561901 12125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:02.562181 12125 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:49:02.562574 12125 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:49:02.562633 12125 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:02.562680 12125 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:49:02.562711 12125 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:02.570432 12125 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.193:44045
I20260806 22:49:02.570498 12299 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.193:44045 every 8 connection(s)
I20260806 22:49:02.570915 12125 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
I20260806 22:49:02.575706 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 12125
I20260806 22:49:02.575867 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/minicluster-data/ts-0/wal/instance
I20260806 22:49:02.576159 18615 external_mini_cluster.cc:1428] Setting key 1fa9a88bc5439f481c85cd99dc8c9775
Aug 06 22:49:02 dist-test-slave-h3ft krb5kdc[11981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056542, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.193@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:49:02.582533 12094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:02.571844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:57787 (local address 127.18.45.254:45249)
0806 22:49:02.571902 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:02.571904 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:02.572232 (+ 328us) server_negotiation.cc:407] Connection header received
0806 22:49:02.572783 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:02.572784 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:02.572800 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:02.572870 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:02.574017 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:02.574669 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:02.576009 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:02.576359 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:02.576445 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:02.579267 (+ 2822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:02.579270 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:02.579270 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:02.579285 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:02.580846 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:02.581355 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:02.581356 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:02.581357 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:02.581438 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:02.581744 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:02.581745 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:02.581745 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:02.581891 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:02.581970 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:02.582313 (+ 343us) server_negotiation.cc:299] Negotiation successful
0806 22:49:02.582423 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 22:49:02.582777 12303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:02.571806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:45249 (local address 127.18.45.193:57787)
0806 22:49:02.572001 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:02.572044 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:02.572677 (+ 633us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:02.572936 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:02.572940 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:02.573611 (+ 671us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:02.573854 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:02.573859 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:02.574786 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:02.574787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:02.575885 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:02.575890 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:02.576480 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:02.576482 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:02.576646 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:02.577317 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:49:02.577328 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:02.579155 (+ 1827us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:02.580969 (+ 1814us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:02.580979 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:02.580982 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:02.581257 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:02.581528 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:02.581529 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:02.581530 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:02.581660 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:02.581969 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:02.581978 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:02.582158 (+ 180us) client_negotiation.cc:769] Sending connection context
0806 22:49:02.582274 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 22:49:02.582476 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":109,"threads_started":1}
I20260806 22:49:02.583528 12301 heartbeater.cc:347] Connected to a master server at 127.18.45.254:45249
I20260806 22:49:02.583684 12301 heartbeater.cc:464] Registering TS with master...
I20260806 22:49:02.583923 12301 heartbeater.cc:511] Master 127.18.45.254:45249 requested a full tablet report, sending...
I20260806 22:49:02.584749 12044 ts_manager.cc:194] Registered new tserver with Master: d6c0ae92ca814f0aa78053903d715a15 (127.18.45.193:44045)
I20260806 22:49:02.585850 12044 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.45.193@KRBTEST.COM'} at 127.18.45.193:57787
WARNING: no policy specified for kudu/127.18.45.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab.
Entry for principal kudu/127.18.45.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab.
I20260806 22:49:02.611119 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.194:0
--local_ip_for_outbound_sockets=127.18.45.194
--webserver_interface=127.18.45.194
--webserver_port=0
--tserver_master_addrs=127.18.45.254:45249
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab
--principal=kudu/127.18.45.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:46339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:49:02.712219 12310 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:02.712520 12310 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:02.712549 12310 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:02.713945 12310 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:02.714011 12310 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:02.714046 12310 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.194
I20260806 22:49:02.715858 12310 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:46339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab
--principal=kudu/127.18.45.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.45.194
--webserver_port=0
--tserver_master_addrs=127.18.45.254:45249
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12310
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.194
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:49:02.716204 12310 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:02.716596 12310 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:02.723404 12316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:02.723393 12320 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:02.723402 12315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:02.723719 12310 server_base.cc:1034] running on GCE node
I20260806 22:49:02.723995 12310 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:02.724311 12310 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:02.725539 12310 hybrid_clock.cc:648] HybridClock initialized: now 1786056542725542 us; error 54 us; skew 500 ppm
Aug 06 22:49:02 dist-test-slave-h3ft krb5kdc[11981](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056542, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:02.729164 12310 init.cc:377] Logged in from keytab as kudu/127.18.45.194@KRBTEST.COM (short username kudu)
I20260806 22:49:02.729849 12310 webserver.cc:460] Webserver started at http://127.18.45.194:36909/ using document root <none> and password file <none>
I20260806 22:49:02.730139 12310 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:02.730216 12310 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:02.730419 12310 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:49:02.731809 12310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/minicluster-data/ts-1/data/instance:
uuid: "dc0cacf626604e0bbdf9fef2a2d8fe4b"
format_stamp: "Formatted at 2026-08-06 22:49:02 on dist-test-slave-h3ft"
server_key: "848c16501126d1817e482385dc09391e"
server_key_iv: "e1c1e9b2f8f66a566078f155480454fb"
server_key_version: "encryptionkey@0"
I20260806 22:49:02.732249 12310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/minicluster-data/ts-1/wal/instance:
uuid: "dc0cacf626604e0bbdf9fef2a2d8fe4b"
format_stamp: "Formatted at 2026-08-06 22:49:02 on dist-test-slave-h3ft"
server_key: "848c16501126d1817e482385dc09391e"
server_key_iv: "e1c1e9b2f8f66a566078f155480454fb"
server_key_version: "encryptionkey@0"
I20260806 22:49:02.733925 12310 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:49:02.735052 12328 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:02.735322 12310 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:49:02.735491 12310 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/minicluster-data/ts-1/wal
uuid: "dc0cacf626604e0bbdf9fef2a2d8fe4b"
format_stamp: "Formatted at 2026-08-06 22:49:02 on dist-test-slave-h3ft"
server_key: "848c16501126d1817e482385dc09391e"
server_key_iv: "e1c1e9b2f8f66a566078f155480454fb"
server_key_version: "encryptionkey@0"
I20260806 22:49:02.735589 12310 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:02.756234 12310 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:02.759377 12310 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:02.759594 12310 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:02.759925 12310 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:49:02.760360 12310 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:49:02.760422 12310 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:02.760486 12310 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:49:02.760521 12310 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:02.768363 12310 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.194:35105
I20260806 22:49:02.768419 12481 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.194:35105 every 8 connection(s)
I20260806 22:49:02.768824 12310 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
I20260806 22:49:02.769860 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 12310
I20260806 22:49:02.770043 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/minicluster-data/ts-1/wal/instance
I20260806 22:49:02.770325 18615 external_mini_cluster.cc:1428] Setting key aea63c7a3b0cfbab546209aff6231334
Aug 06 22:49:02 dist-test-slave-h3ft krb5kdc[11981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056542, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.194@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:49:02.780095 12094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:02.769847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:60435 (local address 127.18.45.254:45249)
0806 22:49:02.769986 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:02.769989 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:02.770299 (+ 310us) server_negotiation.cc:407] Connection header received
0806 22:49:02.771094 (+ 795us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:02.771095 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:02.771109 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:02.771171 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:02.772309 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:02.772888 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:02.773994 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:02.774206 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:02.774305 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:02.776862 (+ 2557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:02.776864 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:02.776864 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:02.776873 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:02.778441 (+ 1568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:02.778901 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:02.778903 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:02.778903 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:02.778975 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:02.779313 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:02.779315 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:02.779315 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:02.779477 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:02.779559 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:02.779877 (+ 318us) server_negotiation.cc:299] Negotiation successful
0806 22:49:02.779984 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260806 22:49:02.780334 12485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:02.769689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:45249 (local address 127.18.45.194:60435)
0806 22:49:02.770114 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:02.770171 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:02.770960 (+ 789us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:02.771259 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:02.771264 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:02.771944 (+ 680us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:02.772178 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:02.772183 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:02.773004 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:02.773006 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:02.773893 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:02.773895 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:02.774329 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:02.774330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:02.774485 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:02.775190 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:49:02.775200 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:02.776754 (+ 1554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:02.778545 (+ 1791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:02.778554 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:02.778557 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:02.778809 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:02.779091 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:02.779092 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:02.779093 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:02.779224 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:02.779559 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:02.779568 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:02.779735 (+ 167us) client_negotiation.cc:769] Sending connection context
0806 22:49:02.779855 (+ 120us) client_negotiation.cc:240] Negotiation successful
0806 22:49:02.780056 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":115,"threads_started":1}
I20260806 22:49:02.780908 12482 heartbeater.cc:347] Connected to a master server at 127.18.45.254:45249
I20260806 22:49:02.781019 12482 heartbeater.cc:464] Registering TS with master...
I20260806 22:49:02.781261 12482 heartbeater.cc:511] Master 127.18.45.254:45249 requested a full tablet report, sending...
I20260806 22:49:02.781814 12044 ts_manager.cc:194] Registered new tserver with Master: dc0cacf626604e0bbdf9fef2a2d8fe4b (127.18.45.194:35105)
I20260806 22:49:02.782262 12044 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.45.194@KRBTEST.COM'} at 127.18.45.194:60435
WARNING: no policy specified for kudu/127.18.45.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab.
Entry for principal kudu/127.18.45.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab.
I20260806 22:49:02.807682 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.195:0
--local_ip_for_outbound_sockets=127.18.45.195
--webserver_interface=127.18.45.195
--webserver_port=0
--tserver_master_addrs=127.18.45.254:45249
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab
--principal=kudu/127.18.45.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:46339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:49:02.916589 12488 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:02.916843 12488 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:02.916870 12488 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:02.918165 12488 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:02.918236 12488 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:02.918268 12488 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.195
I20260806 22:49:02.919972 12488 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:46339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab
--principal=kudu/127.18.45.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.45.195
--webserver_port=0
--tserver_master_addrs=127.18.45.254:45249
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12488
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.195
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:49:02.920280 12488 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:02.920568 12488 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:02.926838 12496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:02.926848 12495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:02.926855 12498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:02.927340 12488 server_base.cc:1034] running on GCE node
I20260806 22:49:02.927625 12488 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:02.927958 12488 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:02.929169 12488 hybrid_clock.cc:648] HybridClock initialized: now 1786056542929143 us; error 45 us; skew 500 ppm
Aug 06 22:49:02 dist-test-slave-h3ft krb5kdc[11981](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056542, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:02.932701 12488 init.cc:377] Logged in from keytab as kudu/127.18.45.195@KRBTEST.COM (short username kudu)
I20260806 22:49:02.933264 12488 webserver.cc:460] Webserver started at http://127.18.45.195:39669/ using document root <none> and password file <none>
I20260806 22:49:02.933565 12488 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:02.933643 12488 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:02.933809 12488 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:49:02.935123 12488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/minicluster-data/ts-2/data/instance:
uuid: "e4020ff9f4894634b83b16d5932a17bc"
format_stamp: "Formatted at 2026-08-06 22:49:02 on dist-test-slave-h3ft"
server_key: "8c8dce17c372ff88d8d6d5a720851874"
server_key_iv: "9b37f35ca44fdec84821cb9aab701505"
server_key_version: "encryptionkey@0"
I20260806 22:49:02.935534 12488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/minicluster-data/ts-2/wal/instance:
uuid: "e4020ff9f4894634b83b16d5932a17bc"
format_stamp: "Formatted at 2026-08-06 22:49:02 on dist-test-slave-h3ft"
server_key: "8c8dce17c372ff88d8d6d5a720851874"
server_key_iv: "9b37f35ca44fdec84821cb9aab701505"
server_key_version: "encryptionkey@0"
I20260806 22:49:02.937132 12488 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 22:49:02.938090 12508 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:02.938385 12488 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:49:02.938526 12488 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/minicluster-data/ts-2/wal
uuid: "e4020ff9f4894634b83b16d5932a17bc"
format_stamp: "Formatted at 2026-08-06 22:49:02 on dist-test-slave-h3ft"
server_key: "8c8dce17c372ff88d8d6d5a720851874"
server_key_iv: "9b37f35ca44fdec84821cb9aab701505"
server_key_version: "encryptionkey@0"
I20260806 22:49:02.938623 12488 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:02.951656 12488 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:02.955148 12488 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:02.955381 12488 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:02.955732 12488 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:49:02.956207 12488 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:49:02.956276 12488 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:02.956331 12488 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:49:02.956363 12488 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:02.964916 12488 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.195:38201
I20260806 22:49:02.964967 12668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.195:38201 every 8 connection(s)
I20260806 22:49:02.965430 12488 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
I20260806 22:49:02.965876 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 12488
I20260806 22:49:02.965999 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/minicluster-data/ts-2/wal/instance
I20260806 22:49:02.966208 18615 external_mini_cluster.cc:1428] Setting key a6a7e43de958d5a2f2fcff8d0aaf325e
Aug 06 22:49:02 dist-test-slave-h3ft krb5kdc[11981](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056542, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.195@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:49:02.977162 12094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:02.966407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.195:32893 (local address 127.18.45.254:45249)
0806 22:49:02.966476 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:02.966479 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:02.966799 (+ 320us) server_negotiation.cc:407] Connection header received
0806 22:49:02.967566 (+ 767us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:02.967568 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:02.967583 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:02.967631 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:02.968827 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:02.969404 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:02.970560 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:02.970817 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:02.970905 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:02.973869 (+ 2964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:02.973870 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:02.973871 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:02.973879 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:02.975524 (+ 1645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:02.976001 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:02.976002 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:02.976003 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:02.976072 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:02.976417 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:02.976418 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:02.976418 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:02.976563 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:02.976629 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:02.976985 (+ 356us) server_negotiation.cc:299] Negotiation successful
0806 22:49:02.977069 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 22:49:02.977329 12674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:02.966365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:45249 (local address 127.18.45.195:32893)
0806 22:49:02.966583 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:02.966647 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:02.967447 (+ 800us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:02.967716 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:02.967721 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:02.968434 (+ 713us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:02.968697 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:02.968702 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:02.969533 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:02.969534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:02.970441 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:02.970444 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:02.970915 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:02.970916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:02.971167 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:02.971870 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:49:02.971881 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:02.973762 (+ 1881us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:02.975624 (+ 1862us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:02.975633 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:02.975636 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:02.975898 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:02.976178 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:02.976179 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:02.976180 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:02.976317 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:02.976634 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:02.976641 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:02.976796 (+ 155us) client_negotiation.cc:769] Sending connection context
0806 22:49:02.976898 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 22:49:02.977060 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":109,"threads_started":1}
I20260806 22:49:02.978005 12672 heartbeater.cc:347] Connected to a master server at 127.18.45.254:45249
I20260806 22:49:02.978164 12672 heartbeater.cc:464] Registering TS with master...
I20260806 22:49:02.978433 12672 heartbeater.cc:511] Master 127.18.45.254:45249 requested a full tablet report, sending...
I20260806 22:49:02.978905 12044 ts_manager.cc:194] Registered new tserver with Master: e4020ff9f4894634b83b16d5932a17bc (127.18.45.195:38201)
I20260806 22:49:02.979396 12044 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.45.195@KRBTEST.COM'} at 127.18.45.195:32893
I20260806 22:49:02.989944 18615 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:49:03.000311 12094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:02.991921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34196 (local address 127.18.45.254:45249)
0806 22:49:02.992006 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:02.992010 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:02.992237 (+ 227us) server_negotiation.cc:407] Connection header received
0806 22:49:02.992273 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:02.992274 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:02.992287 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:02.992391 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:02.993050 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:02.993617 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:02.994431 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:02.994643 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:02.994714 (+ 71us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:02.996755 (+ 2041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:02.996757 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:02.996757 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:02.996766 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:02.998367 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:02.998992 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:02.998993 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:02.998994 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:02.999148 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:02.999492 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:02.999493 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:02.999494 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:02.999711 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:02.999826 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:03.000070 (+ 244us) server_negotiation.cc:299] Negotiation successful
0806 22:49:03.000185 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 22:49:03.002784 12044 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34196:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:49:03.005084 12044 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:49:03.014474 12689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:03.008769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.195:38201 (local address 127.0.0.1:35718)
0806 22:49:03.009354 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:03.009367 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:03.009422 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:03.009839 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:03.009840 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:03.009843 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:49:03.009909 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:03.009912 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:03.011511 (+ 1599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:03.011512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:03.013002 (+ 1490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:03.013006 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:03.013936 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:03.013938 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:03.014086 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:03.014130 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 22:49:03.014265 (+ 135us) client_negotiation.cc:240] Negotiation successful
0806 22:49:03.014364 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"mutex_wait_us":85,"spinlock_wait_cycles":768,"thread_start_us":201,"threads_started":1}
I20260806 22:49:03.014487 12687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:03.008929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35718 (local address 127.18.45.195:38201)
0806 22:49:03.009623 (+ 694us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:03.009628 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:03.009643 (+ 15us) server_negotiation.cc:407] Connection header received
0806 22:49:03.009699 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:03.009701 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:03.009719 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:03.009825 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:49:03.009988 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:03.011335 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:03.013119 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:03.013780 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:03.013886 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:03.013962 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:03.014302 (+ 340us) server_negotiation.cc:299] Negotiation successful
0806 22:49:03.014376 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":636,"thread_start_us":112,"threads_started":1}
I20260806 22:49:03.015370 12686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:03.008766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.194:35105 (local address 127.0.0.1:35438)
0806 22:49:03.009102 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:03.009156 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:03.009244 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:03.009671 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:03.009674 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:03.009678 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:49:03.009803 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:03.009807 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:03.011510 (+ 1703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:03.011511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:03.013002 (+ 1491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:03.013006 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:03.015022 (+ 2016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:03.015024 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:03.015109 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:03.015115 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 22:49:03.015197 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 22:49:03.015251 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"mutex_wait_us":78,"thread_start_us":85,"threads_started":1}
I20260806 22:49:03.015409 12690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:03.009304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38902 (local address 127.18.45.193:44045)
0806 22:49:03.009834 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:03.009838 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:03.009855 (+ 17us) server_negotiation.cc:407] Connection header received
0806 22:49:03.009903 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:03.009906 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:03.009923 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:03.010006 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:49:03.010212 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:03.010977 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:03.012409 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:03.013016 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:03.013120 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:03.013200 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:03.015238 (+ 2038us) server_negotiation.cc:299] Negotiation successful
0806 22:49:03.015311 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":463,"thread_start_us":113,"threads_started":1}
I20260806 22:49:03.015383 12684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:03.009049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.193:44045 (local address 127.0.0.1:38902)
0806 22:49:03.009491 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:03.009500 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:03.009540 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:03.010095 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:03.010096 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:03.010097 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:49:03.010129 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:03.010130 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:03.011152 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:03.011153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:03.012304 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:03.012308 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:03.015059 (+ 2751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:03.015061 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:03.015120 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:03.015123 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:49:03.015188 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 22:49:03.015251 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":430,"mutex_wait_us":38,"thread_start_us":100,"threads_started":1}
I20260806 22:49:03.015753 12688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:03.008937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35438 (local address 127.18.45.194:35105)
0806 22:49:03.009459 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:03.009466 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:03.009479 (+ 13us) server_negotiation.cc:407] Connection header received
0806 22:49:03.009536 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:03.009539 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:03.009560 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:03.009634 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:49:03.009928 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:03.011373 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:03.014007 (+ 2634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:03.014893 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:03.014973 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:03.015064 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:03.015593 (+ 529us) server_negotiation.cc:299] Negotiation successful
0806 22:49:03.015659 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":467,"thread_start_us":110,"threads_started":1}
I20260806 22:49:03.015640 12569 tablet_service.cc:1467] Processing CreateTablet for tablet 5e96d37810b8447b89365f2acdbe9c47 (DEFAULT_TABLE table=test-table [id=bf2b1a2ccc9641dfa57c49dd66570818]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:49:03.016166 12569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e96d37810b8447b89365f2acdbe9c47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:03.016423 12213 tablet_service.cc:1467] Processing CreateTablet for tablet 5e96d37810b8447b89365f2acdbe9c47 (DEFAULT_TABLE table=test-table [id=bf2b1a2ccc9641dfa57c49dd66570818]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:49:03.016444 12391 tablet_service.cc:1467] Processing CreateTablet for tablet 5e96d37810b8447b89365f2acdbe9c47 (DEFAULT_TABLE table=test-table [id=bf2b1a2ccc9641dfa57c49dd66570818]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:49:03.016822 12391 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e96d37810b8447b89365f2acdbe9c47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:03.016878 12213 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e96d37810b8447b89365f2acdbe9c47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:03.021201 12695 tablet_bootstrap.cc:492] T 5e96d37810b8447b89365f2acdbe9c47 P d6c0ae92ca814f0aa78053903d715a15: Bootstrap starting.
I20260806 22:49:03.022070 12695 tablet_bootstrap.cc:654] T 5e96d37810b8447b89365f2acdbe9c47 P d6c0ae92ca814f0aa78053903d715a15: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:03.022728 12695 log.cc:824] T 5e96d37810b8447b89365f2acdbe9c47 P d6c0ae92ca814f0aa78053903d715a15: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:03.023250 12691 tablet_bootstrap.cc:492] T 5e96d37810b8447b89365f2acdbe9c47 P dc0cacf626604e0bbdf9fef2a2d8fe4b: Bootstrap starting.
I20260806 22:49:03.023298 12692 tablet_bootstrap.cc:492] T 5e96d37810b8447b89365f2acdbe9c47 P e4020ff9f4894634b83b16d5932a17bc: Bootstrap starting.
I20260806 22:49:03.024145 12691 tablet_bootstrap.cc:654] T 5e96d37810b8447b89365f2acdbe9c47 P dc0cacf626604e0bbdf9fef2a2d8fe4b: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:03.024189 12692 tablet_bootstrap.cc:654] T 5e96d37810b8447b89365f2acdbe9c47 P e4020ff9f4894634b83b16d5932a17bc: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:03.024185 12695 tablet_bootstrap.cc:492] T 5e96d37810b8447b89365f2acdbe9c47 P d6c0ae92ca814f0aa78053903d715a15: No bootstrap required, opened a new log
I20260806 22:49:03.024273 12695 ts_tablet_manager.cc:1397] T 5e96d37810b8447b89365f2acdbe9c47 P d6c0ae92ca814f0aa78053903d715a15: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:49:03.024766 12691 log.cc:824] T 5e96d37810b8447b89365f2acdbe9c47 P dc0cacf626604e0bbdf9fef2a2d8fe4b: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:03.024988 12692 log.cc:824] T 5e96d37810b8447b89365f2acdbe9c47 P e4020ff9f4894634b83b16d5932a17bc: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:03.025892 12692 tablet_bootstrap.cc:492] T 5e96d37810b8447b89365f2acdbe9c47 P e4020ff9f4894634b83b16d5932a17bc: No bootstrap required, opened a new log
I20260806 22:49:03.025908 12691 tablet_bootstrap.cc:492] T 5e96d37810b8447b89365f2acdbe9c47 P dc0cacf626604e0bbdf9fef2a2d8fe4b: No bootstrap required, opened a new log
I20260806 22:49:03.025990 12692 ts_tablet_manager.cc:1397] T 5e96d37810b8447b89365f2acdbe9c47 P e4020ff9f4894634b83b16d5932a17bc: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:49:03.025990 12691 ts_tablet_manager.cc:1397] T 5e96d37810b8447b89365f2acdbe9c47 P dc0cacf626604e0bbdf9fef2a2d8fe4b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:49:03.027359 12695 raft_consensus.cc:348] T 5e96d37810b8447b89365f2acdbe9c47 P d6c0ae92ca814f0aa78053903d715a15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0cacf626604e0bbdf9fef2a2d8fe4b" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 35105 } } peers { permanent_uuid: "d6c0ae92ca814f0aa78053903d715a15" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 44045 } } peers { permanent_uuid: "e4020ff9f4894634b83b16d5932a17bc" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 38201 } }
I20260806 22:49:03.027632 12695 raft_consensus.cc:374] T 5e96d37810b8447b89365f2acdbe9c47 P d6c0ae92ca814f0aa78053903d715a15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:03.027691 12695 raft_consensus.cc:729] T 5e96d37810b8447b89365f2acdbe9c47 P d6c0ae92ca814f0aa78053903d715a15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6c0ae92ca814f0aa78053903d715a15, State: Initialized, Role: FOLLOWER
I20260806 22:49:03.027871 12695 consensus_queue.cc:260] T 5e96d37810b8447b89365f2acdbe9c47 P d6c0ae92ca814f0aa78053903d715a15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0cacf626604e0bbdf9fef2a2d8fe4b" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 35105 } } peers { permanent_uuid: "d6c0ae92ca814f0aa78053903d715a15" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 44045 } } peers { permanent_uuid: "e4020ff9f4894634b83b16d5932a17bc" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 38201 } }
I20260806 22:49:03.028131 12692 raft_consensus.cc:348] T 5e96d37810b8447b89365f2acdbe9c47 P e4020ff9f4894634b83b16d5932a17bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0cacf626604e0bbdf9fef2a2d8fe4b" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 35105 } } peers { permanent_uuid: "d6c0ae92ca814f0aa78053903d715a15" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 44045 } } peers { permanent_uuid: "e4020ff9f4894634b83b16d5932a17bc" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 38201 } }
I20260806 22:49:03.028296 12695 ts_tablet_manager.cc:1428] T 5e96d37810b8447b89365f2acdbe9c47 P d6c0ae92ca814f0aa78053903d715a15: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260806 22:49:03.028326 12692 raft_consensus.cc:374] T 5e96d37810b8447b89365f2acdbe9c47 P e4020ff9f4894634b83b16d5932a17bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:03.028386 12692 raft_consensus.cc:729] T 5e96d37810b8447b89365f2acdbe9c47 P e4020ff9f4894634b83b16d5932a17bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4020ff9f4894634b83b16d5932a17bc, State: Initialized, Role: FOLLOWER
I20260806 22:49:03.028476 12301 heartbeater.cc:503] Master 127.18.45.254:45249 was elected leader, sending a full tablet report...
I20260806 22:49:03.028512 12692 consensus_queue.cc:260] T 5e96d37810b8447b89365f2acdbe9c47 P e4020ff9f4894634b83b16d5932a17bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0cacf626604e0bbdf9fef2a2d8fe4b" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 35105 } } peers { permanent_uuid: "d6c0ae92ca814f0aa78053903d715a15" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 44045 } } peers { permanent_uuid: "e4020ff9f4894634b83b16d5932a17bc" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 38201 } }
I20260806 22:49:03.028853 12692 ts_tablet_manager.cc:1428] T 5e96d37810b8447b89365f2acdbe9c47 P e4020ff9f4894634b83b16d5932a17bc: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:49:03.028858 12672 heartbeater.cc:503] Master 127.18.45.254:45249 was elected leader, sending a full tablet report...
I20260806 22:49:03.028812 12691 raft_consensus.cc:348] T 5e96d37810b8447b89365f2acdbe9c47 P dc0cacf626604e0bbdf9fef2a2d8fe4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0cacf626604e0bbdf9fef2a2d8fe4b" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 35105 } } peers { permanent_uuid: "d6c0ae92ca814f0aa78053903d715a15" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 44045 } } peers { permanent_uuid: "e4020ff9f4894634b83b16d5932a17bc" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 38201 } }
I20260806 22:49:03.029034 12691 raft_consensus.cc:374] T 5e96d37810b8447b89365f2acdbe9c47 P dc0cacf626604e0bbdf9fef2a2d8fe4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:03.029081 12691 raft_consensus.cc:729] T 5e96d37810b8447b89365f2acdbe9c47 P dc0cacf626604e0bbdf9fef2a2d8fe4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc0cacf626604e0bbdf9fef2a2d8fe4b, State: Initialized, Role: FOLLOWER
I20260806 22:49:03.029237 12691 consensus_queue.cc:260] T 5e96d37810b8447b89365f2acdbe9c47 P dc0cacf626604e0bbdf9fef2a2d8fe4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0cacf626604e0bbdf9fef2a2d8fe4b" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 35105 } } peers { permanent_uuid: "d6c0ae92ca814f0aa78053903d715a15" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 44045 } } peers { permanent_uuid: "e4020ff9f4894634b83b16d5932a17bc" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 38201 } }
I20260806 22:49:03.029620 12691 ts_tablet_manager.cc:1428] T 5e96d37810b8447b89365f2acdbe9c47 P dc0cacf626604e0bbdf9fef2a2d8fe4b: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260806 22:49:03.029860 12482 heartbeater.cc:503] Master 127.18.45.254:45249 was elected leader, sending a full tablet report...
W20260806 22:49:03.071956 12302 tablet.cc:2369] T 5e96d37810b8447b89365f2acdbe9c47 P d6c0ae92ca814f0aa78053903d715a15: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:49:03.216318 12673 tablet.cc:2369] T 5e96d37810b8447b89365f2acdbe9c47 P e4020ff9f4894634b83b16d5932a17bc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 22:49:03.269766 12484 tablet.cc:2369] T 5e96d37810b8447b89365f2acdbe9c47 P dc0cacf626604e0bbdf9fef2a2d8fe4b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:49:03.290215 12700 raft_consensus.cc:482] T 5e96d37810b8447b89365f2acdbe9c47 P d6c0ae92ca814f0aa78053903d715a15 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:49:03.290472 12700 raft_consensus.cc:504] T 5e96d37810b8447b89365f2acdbe9c47 P d6c0ae92ca814f0aa78053903d715a15 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0cacf626604e0bbdf9fef2a2d8fe4b" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 35105 } } peers { permanent_uuid: "d6c0ae92ca814f0aa78053903d715a15" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 44045 } } peers { permanent_uuid: "e4020ff9f4894634b83b16d5932a17bc" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 38201 } }
I20260806 22:49:03.291080 12700 leader_election.cc:288] T 5e96d37810b8447b89365f2acdbe9c47 P d6c0ae92ca814f0aa78053903d715a15 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc0cacf626604e0bbdf9fef2a2d8fe4b (127.18.45.194:35105), e4020ff9f4894634b83b16d5932a17bc (127.18.45.195:38201)
I20260806 22:49:03.295957 12688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:03.291438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:41689 (local address 127.18.45.194:35105)
0806 22:49:03.291519 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:03.291522 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:03.291659 (+ 137us) server_negotiation.cc:407] Connection header received
0806 22:49:03.291767 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:03.291769 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:03.291781 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:03.291862 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:49:03.292186 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:03.293015 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:03.294814 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:03.295425 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:03.295546 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:03.295556 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:03.295817 (+ 261us) server_negotiation.cc:299] Negotiation successful
0806 22:49:03.295865 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 22:49:03.295957 12687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:03.291576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:48049 (local address 127.18.45.195:38201)
0806 22:49:03.291654 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:03.291656 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:03.291674 (+ 18us) server_negotiation.cc:407] Connection header received
0806 22:49:03.291760 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:03.291762 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:03.291775 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:03.291862 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:49:03.292190 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:03.293019 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:03.294819 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:03.295454 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:03.295535 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:03.295550 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:03.295817 (+ 267us) server_negotiation.cc:299] Negotiation successful
0806 22:49:03.295868 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 22:49:03.296000 12707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:03.291335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.194:35105 (local address 127.18.45.193:41689)
0806 22:49:03.291570 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:03.291579 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:03.291687 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:03.291934 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:03.291936 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:03.291939 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:49:03.292090 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:03.292092 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:03.293124 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:03.293126 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:03.294697 (+ 1571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:03.294701 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:03.295588 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:03.295588 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:03.295707 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:03.295717 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 22:49:03.295800 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 22:49:03.295866 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"mutex_wait_us":231,"thread_start_us":44,"threads_started":1}
I20260806 22:49:03.296002 12705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:03.291335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.195:38201 (local address 127.18.45.193:48049)
0806 22:49:03.291562 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:03.291577 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:03.291660 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:03.291940 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:03.291941 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:03.291942 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:49:03.292087 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:03.292089 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:03.293125 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:03.293126 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:03.294697 (+ 1571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:03.294701 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:03.295585 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:03.295586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:03.295707 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:03.295716 (+ 9us) client_negotiation.cc:769] Sending connection context
0806 22:49:03.295800 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 22:49:03.295920 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"mutex_wait_us":150,"thread_start_us":96,"threads_started":1}
I20260806 22:49:03.296537 12417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e96d37810b8447b89365f2acdbe9c47" candidate_uuid: "d6c0ae92ca814f0aa78053903d715a15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc0cacf626604e0bbdf9fef2a2d8fe4b" is_pre_election: true
I20260806 22:49:03.296553 12599 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e96d37810b8447b89365f2acdbe9c47" candidate_uuid: "d6c0ae92ca814f0aa78053903d715a15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4020ff9f4894634b83b16d5932a17bc" is_pre_election: true
I20260806 22:49:03.296763 12599 raft_consensus.cc:2454] T 5e96d37810b8447b89365f2acdbe9c47 P e4020ff9f4894634b83b16d5932a17bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6c0ae92ca814f0aa78053903d715a15 in term 0.
I20260806 22:49:03.296763 12417 raft_consensus.cc:2454] T 5e96d37810b8447b89365f2acdbe9c47 P dc0cacf626604e0bbdf9fef2a2d8fe4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6c0ae92ca814f0aa78053903d715a15 in term 0.
I20260806 22:49:03.297076 12147 leader_election.cc:302] T 5e96d37810b8447b89365f2acdbe9c47 P d6c0ae92ca814f0aa78053903d715a15 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6c0ae92ca814f0aa78053903d715a15, dc0cacf626604e0bbdf9fef2a2d8fe4b; no voters:
I20260806 22:49:03.297262 12700 raft_consensus.cc:2781] T 5e96d37810b8447b89365f2acdbe9c47 P d6c0ae92ca814f0aa78053903d715a15 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:49:03.297350 12700 raft_consensus.cc:482] T 5e96d37810b8447b89365f2acdbe9c47 P d6c0ae92ca814f0aa78053903d715a15 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:49:03.297386 12700 raft_consensus.cc:3037] T 5e96d37810b8447b89365f2acdbe9c47 P d6c0ae92ca814f0aa78053903d715a15 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:03.298334 12700 raft_consensus.cc:504] T 5e96d37810b8447b89365f2acdbe9c47 P d6c0ae92ca814f0aa78053903d715a15 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0cacf626604e0bbdf9fef2a2d8fe4b" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 35105 } } peers { permanent_uuid: "d6c0ae92ca814f0aa78053903d715a15" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 44045 } } peers { permanent_uuid: "e4020ff9f4894634b83b16d5932a17bc" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 38201 } }
I20260806 22:49:03.298578 12700 leader_election.cc:288] T 5e96d37810b8447b89365f2acdbe9c47 P d6c0ae92ca814f0aa78053903d715a15 [CANDIDATE]: Term 1 election: Requested vote from peers dc0cacf626604e0bbdf9fef2a2d8fe4b (127.18.45.194:35105), e4020ff9f4894634b83b16d5932a17bc (127.18.45.195:38201)
I20260806 22:49:03.298683 12417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e96d37810b8447b89365f2acdbe9c47" candidate_uuid: "d6c0ae92ca814f0aa78053903d715a15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc0cacf626604e0bbdf9fef2a2d8fe4b"
I20260806 22:49:03.298707 12599 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e96d37810b8447b89365f2acdbe9c47" candidate_uuid: "d6c0ae92ca814f0aa78053903d715a15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4020ff9f4894634b83b16d5932a17bc"
I20260806 22:49:03.298782 12417 raft_consensus.cc:3037] T 5e96d37810b8447b89365f2acdbe9c47 P dc0cacf626604e0bbdf9fef2a2d8fe4b [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:03.298796 12599 raft_consensus.cc:3037] T 5e96d37810b8447b89365f2acdbe9c47 P e4020ff9f4894634b83b16d5932a17bc [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:03.299787 12599 raft_consensus.cc:2454] T 5e96d37810b8447b89365f2acdbe9c47 P e4020ff9f4894634b83b16d5932a17bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6c0ae92ca814f0aa78053903d715a15 in term 1.
I20260806 22:49:03.299958 12417 raft_consensus.cc:2454] T 5e96d37810b8447b89365f2acdbe9c47 P dc0cacf626604e0bbdf9fef2a2d8fe4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6c0ae92ca814f0aa78053903d715a15 in term 1.
I20260806 22:49:03.299988 12151 leader_election.cc:302] T 5e96d37810b8447b89365f2acdbe9c47 P d6c0ae92ca814f0aa78053903d715a15 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6c0ae92ca814f0aa78053903d715a15, e4020ff9f4894634b83b16d5932a17bc; no voters:
I20260806 22:49:03.300094 12700 raft_consensus.cc:2781] T 5e96d37810b8447b89365f2acdbe9c47 P d6c0ae92ca814f0aa78053903d715a15 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:49:03.300348 12700 raft_consensus.cc:686] T 5e96d37810b8447b89365f2acdbe9c47 P d6c0ae92ca814f0aa78053903d715a15 [term 1 LEADER]: Becoming Leader. State: Replica: d6c0ae92ca814f0aa78053903d715a15, State: Running, Role: LEADER
I20260806 22:49:03.300472 12700 consensus_queue.cc:237] T 5e96d37810b8447b89365f2acdbe9c47 P d6c0ae92ca814f0aa78053903d715a15 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0cacf626604e0bbdf9fef2a2d8fe4b" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 35105 } } peers { permanent_uuid: "d6c0ae92ca814f0aa78053903d715a15" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 44045 } } peers { permanent_uuid: "e4020ff9f4894634b83b16d5932a17bc" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 38201 } }
I20260806 22:49:03.301569 12044 catalog_manager.cc:5495] T 5e96d37810b8447b89365f2acdbe9c47 P d6c0ae92ca814f0aa78053903d715a15 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6c0ae92ca814f0aa78053903d715a15 (127.18.45.193). New cstate: current_term: 1 leader_uuid: "d6c0ae92ca814f0aa78053903d715a15" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc0cacf626604e0bbdf9fef2a2d8fe4b" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 35105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6c0ae92ca814f0aa78053903d715a15" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 44045 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e4020ff9f4894634b83b16d5932a17bc" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 38201 } health_report { overall_health: UNKNOWN } } }
I20260806 22:49:03.376227 12690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:03.373466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38908 (local address 127.18.45.193:44045)
0806 22:49:03.373557 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:03.373559 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:03.373567 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:49:03.373625 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:03.373626 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:03.373656 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:03.373740 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:49:03.373999 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:03.374523 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:03.375355 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:03.375570 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:03.375653 (+ 83us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:03.375842 (+ 189us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:49:03.375960 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:49:03.376087 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:03.376119 (+ 32us) server_negotiation.cc:299] Negotiation successful
0806 22:49:03.376133 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 22:49:03.378845 12599 raft_consensus.cc:1261] T 5e96d37810b8447b89365f2acdbe9c47 P e4020ff9f4894634b83b16d5932a17bc [term 1 FOLLOWER]: Refusing update from remote peer d6c0ae92ca814f0aa78053903d715a15: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:49:03.378895 12417 raft_consensus.cc:1261] T 5e96d37810b8447b89365f2acdbe9c47 P dc0cacf626604e0bbdf9fef2a2d8fe4b [term 1 FOLLOWER]: Refusing update from remote peer d6c0ae92ca814f0aa78053903d715a15: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:49:03.379415 12710 consensus_queue.cc:1035] T 5e96d37810b8447b89365f2acdbe9c47 P d6c0ae92ca814f0aa78053903d715a15 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4020ff9f4894634b83b16d5932a17bc" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 38201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:49:03.379680 12700 consensus_queue.cc:1035] T 5e96d37810b8447b89365f2acdbe9c47 P d6c0ae92ca814f0aa78053903d715a15 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc0cacf626604e0bbdf9fef2a2d8fe4b" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 35105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:49:03.383334 12715 mvcc.cc:205] Tried to move back new op lower bound from 7315687601675829248 to 7315687601359450112. Current Snapshot: MvccSnapshot[applied={T|T < 7315687601675829248 or (T in {7315687601675829248})}]
I20260806 22:49:03.383574 12714 mvcc.cc:205] Tried to move back new op lower bound from 7315687601675829248 to 7315687601359450112. Current Snapshot: MvccSnapshot[applied={T|T < 7315687601675829248}]
I20260806 22:49:03.386135 12041 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34196:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:49:03.386272 12041 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34196:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:49:03.387468 12041 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 504045cfcea74b588d76ad9aaa8f6cc5: Sending DeleteTablet for 3 replicas of tablet 5e96d37810b8447b89365f2acdbe9c47
I20260806 22:49:03.388006 12391 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e96d37810b8447b89365f2acdbe9c47 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:49:03 UTC) from {username='kudu', principal='kudu/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:35438
I20260806 22:49:03.388335 12727 tablet_replica.cc:330] T 5e96d37810b8447b89365f2acdbe9c47 P dc0cacf626604e0bbdf9fef2a2d8fe4b: stopping tablet replica
I20260806 22:49:03.388414 12213 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e96d37810b8447b89365f2acdbe9c47 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:49:03 UTC) from {username='kudu', principal='kudu/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:38902
I20260806 22:49:03.388527 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 12125
I20260806 22:49:03.388636 12727 raft_consensus.cc:2229] T 5e96d37810b8447b89365f2acdbe9c47 P dc0cacf626604e0bbdf9fef2a2d8fe4b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:49:03.396780 12569 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e96d37810b8447b89365f2acdbe9c47 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:49:03 UTC) from {username='kudu', principal='kudu/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:35718
I20260806 22:49:03.397116 12727 raft_consensus.cc:2258] T 5e96d37810b8447b89365f2acdbe9c47 P dc0cacf626604e0bbdf9fef2a2d8fe4b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:49:03.397773 12729 tablet_replica.cc:330] T 5e96d37810b8447b89365f2acdbe9c47 P e4020ff9f4894634b83b16d5932a17bc: stopping tablet replica
I20260806 22:49:03.397964 12729 raft_consensus.cc:2229] T 5e96d37810b8447b89365f2acdbe9c47 P e4020ff9f4894634b83b16d5932a17bc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:49:03.398151 12729 raft_consensus.cc:2258] T 5e96d37810b8447b89365f2acdbe9c47 P e4020ff9f4894634b83b16d5932a17bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:49:03.398296 12727 ts_tablet_manager.cc:1905] T 5e96d37810b8447b89365f2acdbe9c47 P dc0cacf626604e0bbdf9fef2a2d8fe4b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:49:03.398874 12729 ts_tablet_manager.cc:1905] T 5e96d37810b8447b89365f2acdbe9c47 P e4020ff9f4894634b83b16d5932a17bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:49:03.400543 12727 ts_tablet_manager.cc:1918] T 5e96d37810b8447b89365f2acdbe9c47 P dc0cacf626604e0bbdf9fef2a2d8fe4b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:49:03.400717 12727 log.cc:1196] T 5e96d37810b8447b89365f2acdbe9c47 P dc0cacf626604e0bbdf9fef2a2d8fe4b: Deleting WAL directory at /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/minicluster-data/ts-1/wal/wals/5e96d37810b8447b89365f2acdbe9c47
I20260806 22:49:03.401074 12727 ts_tablet_manager.cc:1939] T 5e96d37810b8447b89365f2acdbe9c47 P dc0cacf626604e0bbdf9fef2a2d8fe4b: Deleting consensus metadata
I20260806 22:49:03.401372 12729 ts_tablet_manager.cc:1918] T 5e96d37810b8447b89365f2acdbe9c47 P e4020ff9f4894634b83b16d5932a17bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:49:03.401474 12729 log.cc:1196] T 5e96d37810b8447b89365f2acdbe9c47 P e4020ff9f4894634b83b16d5932a17bc: Deleting WAL directory at /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786056275803658-18615-0/minicluster-data/ts-2/wal/wals/5e96d37810b8447b89365f2acdbe9c47
I20260806 22:49:03.401787 12025 catalog_manager.cc:4841] TS dc0cacf626604e0bbdf9fef2a2d8fe4b (127.18.45.194:35105): tablet 5e96d37810b8447b89365f2acdbe9c47 (table test-table [id=bf2b1a2ccc9641dfa57c49dd66570818]) successfully deleted
I20260806 22:49:03.401823 12729 ts_tablet_manager.cc:1939] T 5e96d37810b8447b89365f2acdbe9c47 P e4020ff9f4894634b83b16d5932a17bc: Deleting consensus metadata
I20260806 22:49:03.402391 12026 catalog_manager.cc:4841] TS e4020ff9f4894634b83b16d5932a17bc (127.18.45.195:38201): tablet 5e96d37810b8447b89365f2acdbe9c47 (table test-table [id=bf2b1a2ccc9641dfa57c49dd66570818]) successfully deleted
W20260806 22:49:03.405855 12025 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.45.193:44045 (error 108)
I20260806 22:49:03.406615 12686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:03.406405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.193:44045 (local address 127.0.0.1:38922)
0806 22:49:03.406496 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:03.406545 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.193:44045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260806 22:49:03.406754 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 12310
W20260806 22:49:03.406817 12025 catalog_manager.cc:4584] TS d6c0ae92ca814f0aa78053903d715a15 (127.18.45.193:44045): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5e96d37810b8447b89365f2acdbe9c47: Network error: Client connection negotiation failed: client connection to 127.18.45.193:44045: connect: Connection refused (error 111)
I20260806 22:49:03.419466 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 12488
I20260806 22:49:03.434329 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 11998
2026-08-06T22:49:03Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2708 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-06T22:49:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:49:03Z Disabled control of system clock
I20260806 22:49:03.478009 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056275803658-18615-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056275803658-18615-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056275803658-18615-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:36961
--webserver_interface=127.18.45.254
--webserver_port=0
--builtin_ntp_servers=127.18.45.212:34179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.45.254:36961
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 22:49:03.578718 12734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:03.579089 12734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:03.579137 12734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:03.580610 12734 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:49:03.580685 12734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:03.580705 12734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:03.580726 12734 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:49:03.580754 12734 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:49:03.582280 12734 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:34179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056275803658-18615-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056275803658-18615-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.45.254:36961
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:36961
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.45.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12734
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056275803658-18615-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:49:03.582607 12734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:03.583077 12734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:03.589272 12744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:03.589311 12746 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:03.589370 12734 server_base.cc:1034] running on GCE node
W20260806 22:49:03.589324 12743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:03.590090 12734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:03.590469 12734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:03.591655 12734 hybrid_clock.cc:648] HybridClock initialized: now 1786056543591615 us; error 65 us; skew 500 ppm
I20260806 22:49:03.593303 12734 webserver.cc:460] Webserver started at http://127.18.45.254:42597/ using document root <none> and password file <none>
I20260806 22:49:03.593655 12734 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:03.593736 12734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:03.593916 12734 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:49:03.595331 12734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056275803658-18615-0/minicluster-data/master-0/data/instance:
uuid: "6fceb0fa8170492abe9bcf4dfb93f92f"
format_stamp: "Formatted at 2026-08-06 22:49:03 on dist-test-slave-h3ft"
server_key: "7c9a3945750780486d9701885e7bd74d"
server_key_iv: "e8c4c8a9cdf718f15637f4f659fffb48"
server_key_version: "encryptionkey@0"
I20260806 22:49:03.595768 12734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056275803658-18615-0/minicluster-data/master-0/wal/instance:
uuid: "6fceb0fa8170492abe9bcf4dfb93f92f"
format_stamp: "Formatted at 2026-08-06 22:49:03 on dist-test-slave-h3ft"
server_key: "7c9a3945750780486d9701885e7bd74d"
server_key_iv: "e8c4c8a9cdf718f15637f4f659fffb48"
server_key_version: "encryptionkey@0"
I20260806 22:49:03.597316 12734 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:49:03.598224 12754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:03.598444 12734 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:49:03.598565 12734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056275803658-18615-0/minicluster-data/master-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056275803658-18615-0/minicluster-data/master-0/wal
uuid: "6fceb0fa8170492abe9bcf4dfb93f92f"
format_stamp: "Formatted at 2026-08-06 22:49:03 on dist-test-slave-h3ft"
server_key: "7c9a3945750780486d9701885e7bd74d"
server_key_iv: "e8c4c8a9cdf718f15637f4f659fffb48"
server_key_version: "encryptionkey@0"
I20260806 22:49:03.598651 12734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056275803658-18615-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056275803658-18615-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056275803658-18615-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:03.615917 12734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:03.616302 12734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:03.616437 12734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:03.621603 12734 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.254:36961
I20260806 22:49:03.621682 12822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.254:36961 every 8 connection(s)
I20260806 22:49:03.622282 12734 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
I20260806 22:49:03.623739 12823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:03.624253 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 12734
I20260806 22:49:03.624363 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056275803658-18615-0/minicluster-data/master-0/wal/instance
I20260806 22:49:03.624655 18615 external_mini_cluster.cc:1428] Setting key 56b0136f5f2daa6247bd2ba27451fd67
I20260806 22:49:03.627852 12823 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6fceb0fa8170492abe9bcf4dfb93f92f: Bootstrap starting.
I20260806 22:49:03.628646 12823 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6fceb0fa8170492abe9bcf4dfb93f92f: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:03.628984 12823 log.cc:824] T 00000000000000000000000000000000 P 6fceb0fa8170492abe9bcf4dfb93f92f: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:03.629804 12823 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6fceb0fa8170492abe9bcf4dfb93f92f: No bootstrap required, opened a new log
I20260806 22:49:03.631707 12827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:03.625551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55498 (local address 127.18.45.254:36961)
0806 22:49:03.625868 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:03.625891 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:03.625941 (+ 50us) server_negotiation.cc:407] Connection header received
0806 22:49:03.626695 (+ 754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:03.626707 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:03.626801 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:03.627070 (+ 269us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:03.627343 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:03.628288 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:03.629244 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:03.629696 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:03.629855 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:03.630763 (+ 908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:03.630767 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:03.630767 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:49:03.631079 (+ 312us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:03.631132 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:49:03.631144 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:03.631218 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:03.631353 (+ 135us) server_negotiation.cc:299] Negotiation successful
0806 22:49:03.631458 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":109,"threads_started":1}
I20260806 22:49:03.632431 12823 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6fceb0fa8170492abe9bcf4dfb93f92f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fceb0fa8170492abe9bcf4dfb93f92f" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 36961 } }
I20260806 22:49:03.632660 12823 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6fceb0fa8170492abe9bcf4dfb93f92f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:03.632717 12823 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6fceb0fa8170492abe9bcf4dfb93f92f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fceb0fa8170492abe9bcf4dfb93f92f, State: Initialized, Role: FOLLOWER
I20260806 22:49:03.632858 12823 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6fceb0fa8170492abe9bcf4dfb93f92f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fceb0fa8170492abe9bcf4dfb93f92f" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 36961 } }
I20260806 22:49:03.632946 12823 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6fceb0fa8170492abe9bcf4dfb93f92f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:49:03.632993 12823 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6fceb0fa8170492abe9bcf4dfb93f92f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:49:03.633045 12823 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6fceb0fa8170492abe9bcf4dfb93f92f [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:03.633704 12823 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6fceb0fa8170492abe9bcf4dfb93f92f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fceb0fa8170492abe9bcf4dfb93f92f" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 36961 } }
I20260806 22:49:03.633826 12823 leader_election.cc:302] T 00000000000000000000000000000000 P 6fceb0fa8170492abe9bcf4dfb93f92f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6fceb0fa8170492abe9bcf4dfb93f92f; no voters:
I20260806 22:49:03.634035 12823 leader_election.cc:288] T 00000000000000000000000000000000 P 6fceb0fa8170492abe9bcf4dfb93f92f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:49:03.634135 12829 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6fceb0fa8170492abe9bcf4dfb93f92f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:49:03.634334 12829 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6fceb0fa8170492abe9bcf4dfb93f92f [term 1 LEADER]: Becoming Leader. State: Replica: 6fceb0fa8170492abe9bcf4dfb93f92f, State: Running, Role: LEADER
I20260806 22:49:03.634434 12823 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6fceb0fa8170492abe9bcf4dfb93f92f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:49:03.634442 12829 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6fceb0fa8170492abe9bcf4dfb93f92f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fceb0fa8170492abe9bcf4dfb93f92f" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 36961 } }
I20260806 22:49:03.634933 12831 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6fceb0fa8170492abe9bcf4dfb93f92f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6fceb0fa8170492abe9bcf4dfb93f92f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fceb0fa8170492abe9bcf4dfb93f92f" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 36961 } } }
I20260806 22:49:03.635074 12831 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6fceb0fa8170492abe9bcf4dfb93f92f [sys.catalog]: This master's current role is: LEADER
I20260806 22:49:03.635265 12832 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6fceb0fa8170492abe9bcf4dfb93f92f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6fceb0fa8170492abe9bcf4dfb93f92f. Latest consensus state: current_term: 1 leader_uuid: "6fceb0fa8170492abe9bcf4dfb93f92f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fceb0fa8170492abe9bcf4dfb93f92f" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 36961 } } }
I20260806 22:49:03.635339 12832 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6fceb0fa8170492abe9bcf4dfb93f92f [sys.catalog]: This master's current role is: LEADER
I20260806 22:49:03.635694 12838 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:49:03.636265 12838 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:49:03.638036 12838 catalog_manager.cc:1347] Generated new cluster ID: 5841ec3fe64341f4959ee168292b3d6f
I20260806 22:49:03.638094 12838 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:49:03.647289 12838 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:49:03.647863 12838 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:49:03.653294 12838 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6fceb0fa8170492abe9bcf4dfb93f92f: Generated new TSK 0
I20260806 22:49:03.653512 12838 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:49:03.661777 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056275803658-18615-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056275803658-18615-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056275803658-18615-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.193:0
--local_ip_for_outbound_sockets=127.18.45.193
--webserver_interface=127.18.45.193
--webserver_port=0
--tserver_master_addrs=127.18.45.254:36961
--builtin_ntp_servers=127.18.45.212:34179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 22:49:03.761395 12857 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:03.761672 12857 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:03.761701 12857 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:03.763159 12857 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:03.763211 12857 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:03.763242 12857 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.193
I20260806 22:49:03.764787 12857 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:34179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056275803658-18615-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056275803658-18615-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.45.193
--webserver_port=0
--tserver_master_addrs=127.18.45.254:36961
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12857
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.193
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056275803658-18615-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:49:03.765127 12857 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:03.765532 12857 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:03.771888 12867 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:03.772025 12857 server_base.cc:1034] running on GCE node
W20260806 22:49:03.771940 12865 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:03.771986 12864 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:03.772433 12857 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:03.772840 12857 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:03.774050 12857 hybrid_clock.cc:648] HybridClock initialized: now 1786056543774022 us; error 47 us; skew 500 ppm
I20260806 22:49:03.775811 12857 webserver.cc:460] Webserver started at http://127.18.45.193:32891/ using document root <none> and password file <none>
I20260806 22:49:03.776170 12857 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:03.776230 12857 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:03.776453 12857 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:49:03.777851 12857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056275803658-18615-0/minicluster-data/ts-0/data/instance:
uuid: "60db83400a78493499d5a8ed3a7c7d1f"
format_stamp: "Formatted at 2026-08-06 22:49:03 on dist-test-slave-h3ft"
server_key: "ea1278020692e5706e8042cb1577572e"
server_key_iv: "337e6081d37431444b2235412591da63"
server_key_version: "encryptionkey@0"
I20260806 22:49:03.778303 12857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056275803658-18615-0/minicluster-data/ts-0/wal/instance:
uuid: "60db83400a78493499d5a8ed3a7c7d1f"
format_stamp: "Formatted at 2026-08-06 22:49:03 on dist-test-slave-h3ft"
server_key: "ea1278020692e5706e8042cb1577572e"
server_key_iv: "337e6081d37431444b2235412591da63"
server_key_version: "encryptionkey@0"
I20260806 22:49:03.780206 12857 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260806 22:49:03.781148 12875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:03.781351 12857 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:49:03.781474 12857 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056275803658-18615-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056275803658-18615-0/minicluster-data/ts-0/wal
uuid: "60db83400a78493499d5a8ed3a7c7d1f"
format_stamp: "Formatted at 2026-08-06 22:49:03 on dist-test-slave-h3ft"
server_key: "ea1278020692e5706e8042cb1577572e"
server_key_iv: "337e6081d37431444b2235412591da63"
server_key_version: "encryptionkey@0"
I20260806 22:49:03.781566 12857 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056275803658-18615-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056275803658-18615-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056275803658-18615-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:03.794204 12857 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:03.794739 12857 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:03.794958 12857 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:03.795421 12857 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:49:03.795915 12857 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:49:03.795984 12857 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:03.796037 12857 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:49:03.796085 12857 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:03.804636 12857 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.193:36131
I20260806 22:49:03.804687 13031 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.193:36131 every 8 connection(s)
I20260806 22:49:03.805176 12857 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
I20260806 22:49:03.809845 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 12857
I20260806 22:49:03.810040 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056275803658-18615-0/minicluster-data/ts-0/wal/instance
I20260806 22:49:03.810316 18615 external_mini_cluster.cc:1428] Setting key c03852282cb8cf5a44aa68e13f5d7d04
I20260806 22:49:03.811746 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056275803658-18615-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056275803658-18615-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056275803658-18615-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.194:0
--local_ip_for_outbound_sockets=127.18.45.194
--webserver_interface=127.18.45.194
--webserver_port=0
--tserver_master_addrs=127.18.45.254:36961
--builtin_ntp_servers=127.18.45.212:34179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 22:49:03.811807 12827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:03.806072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:53001 (local address 127.18.45.254:36961)
0806 22:49:03.806166 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:03.806168 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:03.806465 (+ 297us) server_negotiation.cc:407] Connection header received
0806 22:49:03.807170 (+ 705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:03.807172 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:03.807196 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:03.807262 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:03.808091 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:03.808693 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:03.809847 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:03.810156 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:03.810226 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:03.811436 (+ 1210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:03.811438 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:03.811439 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:49:03.811445 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:03.811467 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:49:03.811473 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:03.811545 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:03.811661 (+ 116us) server_negotiation.cc:299] Negotiation successful
0806 22:49:03.811709 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 22:49:03.812064 13034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:03.806055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:36961 (local address 127.18.45.193:53001)
0806 22:49:03.806313 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:03.806356 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:03.807041 (+ 685us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:03.807349 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:03.807353 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:03.807363 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:03.807985 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:03.807990 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:03.808807 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:03.808808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:03.809740 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:03.809743 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:03.810260 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:03.810261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:03.810457 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:03.811251 (+ 794us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:49:03.811251 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:03.811291 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:49:03.811321 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:03.811547 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:03.811554 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:03.811559 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:49:03.811664 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 22:49:03.811800 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":114,"threads_started":1}
I20260806 22:49:03.814618 13032 heartbeater.cc:347] Connected to a master server at 127.18.45.254:36961
I20260806 22:49:03.814783 13032 heartbeater.cc:464] Registering TS with master...
I20260806 22:49:03.815137 13032 heartbeater.cc:511] Master 127.18.45.254:36961 requested a full tablet report, sending...
I20260806 22:49:03.816090 12775 ts_manager.cc:194] Registered new tserver with Master: 60db83400a78493499d5a8ed3a7c7d1f (127.18.45.193:36131)
I20260806 22:49:03.817257 12775 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.45.193:53001
W20260806 22:49:03.915719 13035 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:03.915987 13035 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:03.916014 13035 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:03.917207 13035 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:03.917243 13035 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:03.917268 13035 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.194
I20260806 22:49:03.918618 13035 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:34179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056275803658-18615-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056275803658-18615-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.45.194
--webserver_port=0
--tserver_master_addrs=127.18.45.254:36961
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13035
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.194
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056275803658-18615-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:49:03.918857 13035 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:03.919263 13035 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:03.925396 13047 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:03.925446 13042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:03.925396 13044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:03.925725 13035 server_base.cc:1034] running on GCE node
I20260806 22:49:03.926054 13035 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:03.926369 13035 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:03.927537 13035 hybrid_clock.cc:648] HybridClock initialized: now 1786056543927527 us; error 40 us; skew 500 ppm
I20260806 22:49:03.929307 13035 webserver.cc:460] Webserver started at http://127.18.45.194:45017/ using document root <none> and password file <none>
I20260806 22:49:03.929651 13035 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:03.929749 13035 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:03.929939 13035 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:49:03.931393 13035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056275803658-18615-0/minicluster-data/ts-1/data/instance:
uuid: "f6512870b58747aeae6dd11f0bae6e8e"
format_stamp: "Formatted at 2026-08-06 22:49:03 on dist-test-slave-h3ft"
server_key: "f87e1b1e0880b057137b7fc4d4e5147a"
server_key_iv: "4fb98b4680ed3c68cc10f6c9b194a369"
server_key_version: "encryptionkey@0"
I20260806 22:49:03.931843 13035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056275803658-18615-0/minicluster-data/ts-1/wal/instance:
uuid: "f6512870b58747aeae6dd11f0bae6e8e"
format_stamp: "Formatted at 2026-08-06 22:49:03 on dist-test-slave-h3ft"
server_key: "f87e1b1e0880b057137b7fc4d4e5147a"
server_key_iv: "4fb98b4680ed3c68cc10f6c9b194a369"
server_key_version: "encryptionkey@0"
I20260806 22:49:03.933631 13035 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 22:49:03.934900 13055 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:03.935286 13035 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:49:03.935436 13035 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056275803658-18615-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056275803658-18615-0/minicluster-data/ts-1/wal
uuid: "f6512870b58747aeae6dd11f0bae6e8e"
format_stamp: "Formatted at 2026-08-06 22:49:03 on dist-test-slave-h3ft"
server_key: "f87e1b1e0880b057137b7fc4d4e5147a"
server_key_iv: "4fb98b4680ed3c68cc10f6c9b194a369"
server_key_version: "encryptionkey@0"
I20260806 22:49:03.935559 13035 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056275803658-18615-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056275803658-18615-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056275803658-18615-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:03.967645 13035 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:03.968140 13035 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:03.968348 13035 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:03.968690 13035 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:49:03.969161 13035 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:49:03.969225 13035 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:03.969277 13035 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:49:03.969326 13035 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:03.977378 13035 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.194:45259
I20260806 22:49:03.977437 13215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.194:45259 every 8 connection(s)
I20260806 22:49:03.977890 13035 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
I20260806 22:49:03.981266 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 13035
I20260806 22:49:03.981403 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056275803658-18615-0/minicluster-data/ts-1/wal/instance
I20260806 22:49:03.981659 18615 external_mini_cluster.cc:1428] Setting key d254313422aa9a7d395155eefecf3e50
I20260806 22:49:03.983193 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056275803658-18615-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056275803658-18615-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056275803658-18615-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.195:0
--local_ip_for_outbound_sockets=127.18.45.195
--webserver_interface=127.18.45.195
--webserver_port=0
--tserver_master_addrs=127.18.45.254:36961
--builtin_ntp_servers=127.18.45.212:34179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 22:49:03.985426 12827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:03.978872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:34037 (local address 127.18.45.254:36961)
0806 22:49:03.978955 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:03.978957 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:03.979288 (+ 331us) server_negotiation.cc:407] Connection header received
0806 22:49:03.979921 (+ 633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:03.979923 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:03.979937 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:03.980026 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:03.980911 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:03.981871 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:03.983483 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:03.983733 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:03.983830 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:03.984957 (+ 1127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:03.984959 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:03.984959 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:49:03.984967 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:03.984990 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:49:03.984996 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:03.985061 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:03.985219 (+ 158us) server_negotiation.cc:299] Negotiation successful
0806 22:49:03.985261 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 22:49:03.985749 13218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:03.978841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:36961 (local address 127.18.45.194:34037)
0806 22:49:03.979117 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:03.979171 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:03.979821 (+ 650us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:03.980150 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:03.980155 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:03.980166 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:03.980819 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:03.980824 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:03.982012 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:03.982013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:03.983303 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:03.983308 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:03.983842 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:03.983843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:03.984029 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:03.984761 (+ 732us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:49:03.984762 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:03.984799 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:49:03.984828 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:03.985109 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:03.985118 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:03.985122 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:49:03.985214 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 22:49:03.985376 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":133,"threads_started":1}
I20260806 22:49:03.986419 13216 heartbeater.cc:347] Connected to a master server at 127.18.45.254:36961
I20260806 22:49:03.986557 13216 heartbeater.cc:464] Registering TS with master...
I20260806 22:49:03.986825 13216 heartbeater.cc:511] Master 127.18.45.254:36961 requested a full tablet report, sending...
I20260806 22:49:03.987438 12775 ts_manager.cc:194] Registered new tserver with Master: f6512870b58747aeae6dd11f0bae6e8e (127.18.45.194:45259)
I20260806 22:49:03.987924 12775 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.45.194:34037
W20260806 22:49:04.094732 13221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:04.095135 13221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:04.095189 13221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:04.096608 13221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:04.096665 13221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:04.096695 13221 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.195
I20260806 22:49:04.098253 13221 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:34179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056275803658-18615-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056275803658-18615-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.45.195
--webserver_port=0
--tserver_master_addrs=127.18.45.254:36961
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13221
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.195
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056275803658-18615-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:49:04.098570 13221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:04.098975 13221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:04.105808 13229 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:04.105839 13228 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:04.105813 13233 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:04.106164 13221 server_base.cc:1034] running on GCE node
I20260806 22:49:04.106485 13221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:04.106828 13221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:04.108050 13221 hybrid_clock.cc:648] HybridClock initialized: now 1786056544108044 us; error 59 us; skew 500 ppm
I20260806 22:49:04.110018 13221 webserver.cc:460] Webserver started at http://127.18.45.195:42877/ using document root <none> and password file <none>
I20260806 22:49:04.110472 13221 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:04.110549 13221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:04.110736 13221 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:49:04.112334 13221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056275803658-18615-0/minicluster-data/ts-2/data/instance:
uuid: "f5d388a777fb40858d7ec7e12c1f2c39"
format_stamp: "Formatted at 2026-08-06 22:49:04 on dist-test-slave-h3ft"
server_key: "c2901a48eb363c7e42140c72a81b4a40"
server_key_iv: "73818219f4b27596eb14d08740e25323"
server_key_version: "encryptionkey@0"
I20260806 22:49:04.112808 13221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056275803658-18615-0/minicluster-data/ts-2/wal/instance:
uuid: "f5d388a777fb40858d7ec7e12c1f2c39"
format_stamp: "Formatted at 2026-08-06 22:49:04 on dist-test-slave-h3ft"
server_key: "c2901a48eb363c7e42140c72a81b4a40"
server_key_iv: "73818219f4b27596eb14d08740e25323"
server_key_version: "encryptionkey@0"
I20260806 22:49:04.114564 13221 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260806 22:49:04.115790 13243 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:04.116096 13221 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:49:04.116205 13221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056275803658-18615-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056275803658-18615-0/minicluster-data/ts-2/wal
uuid: "f5d388a777fb40858d7ec7e12c1f2c39"
format_stamp: "Formatted at 2026-08-06 22:49:04 on dist-test-slave-h3ft"
server_key: "c2901a48eb363c7e42140c72a81b4a40"
server_key_iv: "73818219f4b27596eb14d08740e25323"
server_key_version: "encryptionkey@0"
I20260806 22:49:04.116291 13221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056275803658-18615-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056275803658-18615-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056275803658-18615-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:04.124943 13221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:04.125295 13221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:04.125424 13221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:04.125658 13221 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:49:04.126024 13221 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:49:04.126058 13221 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:04.126083 13221 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:49:04.126096 13221 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:04.133677 13221 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.195:41125
I20260806 22:49:04.133778 13401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.195:41125 every 8 connection(s)
I20260806 22:49:04.134218 13221 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
I20260806 22:49:04.141324 12827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:04.135383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.195:57553 (local address 127.18.45.254:36961)
0806 22:49:04.135484 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:04.135487 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:04.135790 (+ 303us) server_negotiation.cc:407] Connection header received
0806 22:49:04.136640 (+ 850us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:04.136642 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:04.136661 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:04.136720 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:04.137603 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:04.138216 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:04.139546 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:04.139775 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:04.139871 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:04.140963 (+ 1092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:04.140965 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:04.140966 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:49:04.140972 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:04.140997 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:49:04.141002 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:04.141071 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:04.141182 (+ 111us) server_negotiation.cc:299] Negotiation successful
0806 22:49:04.141219 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 22:49:04.141625 13408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:04.135383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:36961 (local address 127.18.45.195:57553)
0806 22:49:04.135632 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:04.135679 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:04.136445 (+ 766us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:04.136814 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:04.136821 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:04.136839 (+ 18us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:04.137492 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:04.137498 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:04.138336 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:04.138337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:04.139427 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:04.139430 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:04.139887 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:04.139888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:04.140052 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:04.140773 (+ 721us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:49:04.140774 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:04.140811 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:49:04.140840 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:04.141072 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:04.141080 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:04.141085 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:49:04.141184 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 22:49:04.141311 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":84,"threads_started":1}
I20260806 22:49:04.142261 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 13221
I20260806 22:49:04.142336 13402 heartbeater.cc:347] Connected to a master server at 127.18.45.254:36961
I20260806 22:49:04.142397 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056275803658-18615-0/minicluster-data/ts-2/wal/instance
I20260806 22:49:04.142473 13402 heartbeater.cc:464] Registering TS with master...
I20260806 22:49:04.142652 18615 external_mini_cluster.cc:1428] Setting key e8ba3062c11c1654683e26588231606a
I20260806 22:49:04.142746 13402 heartbeater.cc:511] Master 127.18.45.254:36961 requested a full tablet report, sending...
I20260806 22:49:04.143252 12775 ts_manager.cc:194] Registered new tserver with Master: f5d388a777fb40858d7ec7e12c1f2c39 (127.18.45.195:41125)
I20260806 22:49:04.143833 12775 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.45.195:57553
I20260806 22:49:04.143924 18615 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:49:04.149385 12827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:04.145560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55500 (local address 127.18.45.254:36961)
0806 22:49:04.145625 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:04.145628 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:04.145792 (+ 164us) server_negotiation.cc:407] Connection header received
0806 22:49:04.145865 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:04.145866 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:04.145883 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:04.145943 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:04.146307 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:04.146825 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:04.147729 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:04.147962 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:04.148054 (+ 92us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:04.149004 (+ 950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:04.149006 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:04.149007 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:49:04.149012 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:04.149031 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:49:04.149043 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:04.149167 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:04.149264 (+ 97us) server_negotiation.cc:299] Negotiation successful
0806 22:49:04.149277 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 22:49:04.151718 12775 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55500:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:49:04.154047 12775 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:49:04.161160 13424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:04.157188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37664 (local address 127.18.45.194:45259)
0806 22:49:04.157477 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:04.157484 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:04.157499 (+ 15us) server_negotiation.cc:407] Connection header received
0806 22:49:04.157550 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:04.157552 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:04.157570 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:04.157690 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:49:04.157975 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:04.158929 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:04.160188 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:04.160721 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:04.160809 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:04.160931 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:04.161010 (+ 79us) server_negotiation.cc:299] Negotiation successful
0806 22:49:04.161055 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":131,"threads_started":1}
I20260806 22:49:04.161144 13421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:04.156906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.194:45259 (local address 127.0.0.1:37664)
0806 22:49:04.157113 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:04.157169 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:04.157256 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:04.157794 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:04.157798 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:04.157801 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:49:04.157885 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:04.157887 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:04.159049 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:04.159051 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:04.160096 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:04.160099 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:04.160821 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:04.160822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:04.160903 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:04.160917 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 22:49:04.160996 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 22:49:04.161058 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":68,"threads_started":1}
I20260806 22:49:04.162014 13129 tablet_service.cc:1467] Processing CreateTablet for tablet 0552892f92ac466bab43afe5e17fd919 (DEFAULT_TABLE table=test-table [id=6deefc68b1c447fa91ce9a693a8056e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:49:04.162571 13129 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0552892f92ac466bab43afe5e17fd919. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:04.162627 13425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:04.157217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44366 (local address 127.18.45.195:41125)
0806 22:49:04.158076 (+ 859us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:04.158080 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:04.158091 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:49:04.158137 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:04.158139 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:04.158155 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:04.158238 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:49:04.158472 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:04.159620 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:04.161046 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:04.162017 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:04.162098 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:04.162194 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:04.162472 (+ 278us) server_negotiation.cc:299] Negotiation successful
0806 22:49:04.162526 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":804,"thread_start_us":109,"threads_started":1}
I20260806 22:49:04.162839 13426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:04.157469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52096 (local address 127.18.45.193:36131)
0806 22:49:04.157743 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:04.157749 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:04.158008 (+ 259us) server_negotiation.cc:407] Connection header received
0806 22:49:04.158072 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:04.158079 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:04.158096 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:04.158193 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:49:04.158336 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:04.159563 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:04.161045 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:04.162268 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:04.162340 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:04.162435 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:04.162674 (+ 239us) server_negotiation.cc:299] Negotiation successful
0806 22:49:04.162730 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":152,"threads_started":1}
I20260806 22:49:04.162982 13423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:04.156948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.193:36131 (local address 127.0.0.1:52096)
0806 22:49:04.157920 (+ 972us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:04.157929 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:04.157989 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:04.158189 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:04.158190 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:04.158193 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:49:04.158261 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:04.158263 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:04.159727 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:04.159728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:04.160924 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:04.160928 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:04.162474 (+ 1546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:04.162475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:04.162527 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:04.162531 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:49:04.162584 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 22:49:04.162623 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":952,"mutex_wait_us":71,"thread_start_us":99,"threads_started":1}
I20260806 22:49:04.162600 13422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:04.156917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.195:41125 (local address 127.0.0.1:44366)
0806 22:49:04.157762 (+ 845us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:04.157774 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:04.157829 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:04.158322 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:04.158323 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:04.158326 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:49:04.158371 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:04.158373 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:04.159719 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:04.159720 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:04.160924 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:04.160928 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:04.162239 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:04.162241 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:04.162372 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:04.162377 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:49:04.162441 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 22:49:04.162492 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":811,"mutex_wait_us":6,"thread_start_us":106,"threads_started":1}
I20260806 22:49:04.163915 12941 tablet_service.cc:1467] Processing CreateTablet for tablet 0552892f92ac466bab43afe5e17fd919 (DEFAULT_TABLE table=test-table [id=6deefc68b1c447fa91ce9a693a8056e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:49:04.164345 12941 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0552892f92ac466bab43afe5e17fd919. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:04.166960 13427 tablet_bootstrap.cc:492] T 0552892f92ac466bab43afe5e17fd919 P f6512870b58747aeae6dd11f0bae6e8e: Bootstrap starting.
I20260806 22:49:04.167853 13427 tablet_bootstrap.cc:654] T 0552892f92ac466bab43afe5e17fd919 P f6512870b58747aeae6dd11f0bae6e8e: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:04.168174 13429 tablet_bootstrap.cc:492] T 0552892f92ac466bab43afe5e17fd919 P 60db83400a78493499d5a8ed3a7c7d1f: Bootstrap starting.
I20260806 22:49:04.168417 13427 log.cc:824] T 0552892f92ac466bab43afe5e17fd919 P f6512870b58747aeae6dd11f0bae6e8e: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:04.168599 13311 tablet_service.cc:1467] Processing CreateTablet for tablet 0552892f92ac466bab43afe5e17fd919 (DEFAULT_TABLE table=test-table [id=6deefc68b1c447fa91ce9a693a8056e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:49:04.169042 13311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0552892f92ac466bab43afe5e17fd919. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:04.169260 13429 tablet_bootstrap.cc:654] T 0552892f92ac466bab43afe5e17fd919 P 60db83400a78493499d5a8ed3a7c7d1f: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:04.169377 13427 tablet_bootstrap.cc:492] T 0552892f92ac466bab43afe5e17fd919 P f6512870b58747aeae6dd11f0bae6e8e: No bootstrap required, opened a new log
I20260806 22:49:04.169483 13427 ts_tablet_manager.cc:1397] T 0552892f92ac466bab43afe5e17fd919 P f6512870b58747aeae6dd11f0bae6e8e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:49:04.169710 13429 log.cc:824] T 0552892f92ac466bab43afe5e17fd919 P 60db83400a78493499d5a8ed3a7c7d1f: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:04.170634 13429 tablet_bootstrap.cc:492] T 0552892f92ac466bab43afe5e17fd919 P 60db83400a78493499d5a8ed3a7c7d1f: No bootstrap required, opened a new log
I20260806 22:49:04.170714 13429 ts_tablet_manager.cc:1397] T 0552892f92ac466bab43afe5e17fd919 P 60db83400a78493499d5a8ed3a7c7d1f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:49:04.172294 13432 tablet_bootstrap.cc:492] T 0552892f92ac466bab43afe5e17fd919 P f5d388a777fb40858d7ec7e12c1f2c39: Bootstrap starting.
I20260806 22:49:04.172830 13427 raft_consensus.cc:348] T 0552892f92ac466bab43afe5e17fd919 P f6512870b58747aeae6dd11f0bae6e8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db83400a78493499d5a8ed3a7c7d1f" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 36131 } } peers { permanent_uuid: "f6512870b58747aeae6dd11f0bae6e8e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45259 } } peers { permanent_uuid: "f5d388a777fb40858d7ec7e12c1f2c39" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 41125 } }
I20260806 22:49:04.173053 13427 raft_consensus.cc:374] T 0552892f92ac466bab43afe5e17fd919 P f6512870b58747aeae6dd11f0bae6e8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:04.173095 13427 raft_consensus.cc:729] T 0552892f92ac466bab43afe5e17fd919 P f6512870b58747aeae6dd11f0bae6e8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6512870b58747aeae6dd11f0bae6e8e, State: Initialized, Role: FOLLOWER
I20260806 22:49:04.173122 13432 tablet_bootstrap.cc:654] T 0552892f92ac466bab43afe5e17fd919 P f5d388a777fb40858d7ec7e12c1f2c39: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:04.173280 13427 consensus_queue.cc:260] T 0552892f92ac466bab43afe5e17fd919 P f6512870b58747aeae6dd11f0bae6e8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db83400a78493499d5a8ed3a7c7d1f" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 36131 } } peers { permanent_uuid: "f6512870b58747aeae6dd11f0bae6e8e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45259 } } peers { permanent_uuid: "f5d388a777fb40858d7ec7e12c1f2c39" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 41125 } }
I20260806 22:49:04.173504 13432 log.cc:824] T 0552892f92ac466bab43afe5e17fd919 P f5d388a777fb40858d7ec7e12c1f2c39: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:04.173704 13216 heartbeater.cc:503] Master 127.18.45.254:36961 was elected leader, sending a full tablet report...
I20260806 22:49:04.173624 13427 ts_tablet_manager.cc:1428] T 0552892f92ac466bab43afe5e17fd919 P f6512870b58747aeae6dd11f0bae6e8e: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260806 22:49:04.174122 13429 raft_consensus.cc:348] T 0552892f92ac466bab43afe5e17fd919 P 60db83400a78493499d5a8ed3a7c7d1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db83400a78493499d5a8ed3a7c7d1f" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 36131 } } peers { permanent_uuid: "f6512870b58747aeae6dd11f0bae6e8e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45259 } } peers { permanent_uuid: "f5d388a777fb40858d7ec7e12c1f2c39" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 41125 } }
I20260806 22:49:04.174360 13429 raft_consensus.cc:374] T 0552892f92ac466bab43afe5e17fd919 P 60db83400a78493499d5a8ed3a7c7d1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:04.174407 13429 raft_consensus.cc:729] T 0552892f92ac466bab43afe5e17fd919 P 60db83400a78493499d5a8ed3a7c7d1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60db83400a78493499d5a8ed3a7c7d1f, State: Initialized, Role: FOLLOWER
I20260806 22:49:04.174530 13429 consensus_queue.cc:260] T 0552892f92ac466bab43afe5e17fd919 P 60db83400a78493499d5a8ed3a7c7d1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db83400a78493499d5a8ed3a7c7d1f" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 36131 } } peers { permanent_uuid: "f6512870b58747aeae6dd11f0bae6e8e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45259 } } peers { permanent_uuid: "f5d388a777fb40858d7ec7e12c1f2c39" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 41125 } }
I20260806 22:49:04.174634 13432 tablet_bootstrap.cc:492] T 0552892f92ac466bab43afe5e17fd919 P f5d388a777fb40858d7ec7e12c1f2c39: No bootstrap required, opened a new log
I20260806 22:49:04.174841 13429 ts_tablet_manager.cc:1428] T 0552892f92ac466bab43afe5e17fd919 P 60db83400a78493499d5a8ed3a7c7d1f: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260806 22:49:04.174939 13032 heartbeater.cc:503] Master 127.18.45.254:36961 was elected leader, sending a full tablet report...
I20260806 22:49:04.175027 13432 ts_tablet_manager.cc:1397] T 0552892f92ac466bab43afe5e17fd919 P f5d388a777fb40858d7ec7e12c1f2c39: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.001s
I20260806 22:49:04.177282 13432 raft_consensus.cc:348] T 0552892f92ac466bab43afe5e17fd919 P f5d388a777fb40858d7ec7e12c1f2c39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db83400a78493499d5a8ed3a7c7d1f" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 36131 } } peers { permanent_uuid: "f6512870b58747aeae6dd11f0bae6e8e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45259 } } peers { permanent_uuid: "f5d388a777fb40858d7ec7e12c1f2c39" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 41125 } }
I20260806 22:49:04.177419 13432 raft_consensus.cc:374] T 0552892f92ac466bab43afe5e17fd919 P f5d388a777fb40858d7ec7e12c1f2c39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:04.177443 13432 raft_consensus.cc:729] T 0552892f92ac466bab43afe5e17fd919 P f5d388a777fb40858d7ec7e12c1f2c39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5d388a777fb40858d7ec7e12c1f2c39, State: Initialized, Role: FOLLOWER
I20260806 22:49:04.177556 13432 consensus_queue.cc:260] T 0552892f92ac466bab43afe5e17fd919 P f5d388a777fb40858d7ec7e12c1f2c39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db83400a78493499d5a8ed3a7c7d1f" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 36131 } } peers { permanent_uuid: "f6512870b58747aeae6dd11f0bae6e8e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45259 } } peers { permanent_uuid: "f5d388a777fb40858d7ec7e12c1f2c39" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 41125 } }
I20260806 22:49:04.177830 13432 ts_tablet_manager.cc:1428] T 0552892f92ac466bab43afe5e17fd919 P f5d388a777fb40858d7ec7e12c1f2c39: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 22:49:04.177960 13402 heartbeater.cc:503] Master 127.18.45.254:36961 was elected leader, sending a full tablet report...
W20260806 22:49:04.228652 13217 tablet.cc:2369] T 0552892f92ac466bab43afe5e17fd919 P f6512870b58747aeae6dd11f0bae6e8e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:49:04.244132 13433 raft_consensus.cc:482] T 0552892f92ac466bab43afe5e17fd919 P f6512870b58747aeae6dd11f0bae6e8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:49:04.244403 13433 raft_consensus.cc:504] T 0552892f92ac466bab43afe5e17fd919 P f6512870b58747aeae6dd11f0bae6e8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db83400a78493499d5a8ed3a7c7d1f" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 36131 } } peers { permanent_uuid: "f6512870b58747aeae6dd11f0bae6e8e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45259 } } peers { permanent_uuid: "f5d388a777fb40858d7ec7e12c1f2c39" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 41125 } }
I20260806 22:49:04.245042 13433 leader_election.cc:288] T 0552892f92ac466bab43afe5e17fd919 P f6512870b58747aeae6dd11f0bae6e8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60db83400a78493499d5a8ed3a7c7d1f (127.18.45.193:36131), f5d388a777fb40858d7ec7e12c1f2c39 (127.18.45.195:41125)
I20260806 22:49:04.249780 13425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:04.245438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:35897 (local address 127.18.45.195:41125)
0806 22:49:04.245551 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:04.245553 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:04.245565 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:49:04.245613 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:04.245615 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:04.245627 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:04.245692 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:49:04.246018 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:04.246860 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:04.248674 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:04.249284 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:04.249368 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:04.249379 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:04.249620 (+ 241us) server_negotiation.cc:299] Negotiation successful
0806 22:49:04.249686 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 22:49:04.249800 13426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:04.245553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:52655 (local address 127.18.45.193:36131)
0806 22:49:04.245608 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:04.245611 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:04.245737 (+ 126us) server_negotiation.cc:407] Connection header received
0806 22:49:04.245799 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:04.245802 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:04.245819 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:04.245865 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:49:04.246079 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:04.246858 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:04.248675 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:04.249282 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:04.249368 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:04.249379 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:04.249659 (+ 280us) server_negotiation.cc:299] Negotiation successful
0806 22:49:04.249721 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 22:49:04.249800 13218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:04.245295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.195:41125 (local address 127.18.45.194:35897)
0806 22:49:04.245364 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:04.245380 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:04.245446 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:04.245732 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:04.245734 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:04.245736 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:49:04.245846 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:04.245869 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:04.247028 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:04.247029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:04.248553 (+ 1524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:04.248557 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:04.249402 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:04.249404 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:04.249527 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:04.249532 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:49:04.249611 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 22:49:04.249696 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":108}
I20260806 22:49:04.250298 13439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:04.245354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.193:36131 (local address 127.18.45.194:52655)
0806 22:49:04.245613 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:04.245633 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:04.245732 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:04.245898 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:04.245899 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:04.245901 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:49:04.245991 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:04.245993 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:04.247028 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:04.247029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:04.248553 (+ 1524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:04.248557 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:04.249405 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:04.249406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:04.249529 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:04.249532 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:49:04.250147 (+ 615us) client_negotiation.cc:240] Negotiation successful
0806 22:49:04.250208 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"mutex_wait_us":97,"thread_start_us":185,"threads_started":1}
I20260806 22:49:04.250357 13340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0552892f92ac466bab43afe5e17fd919" candidate_uuid: "f6512870b58747aeae6dd11f0bae6e8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5d388a777fb40858d7ec7e12c1f2c39" is_pre_election: true
I20260806 22:49:04.250622 13340 raft_consensus.cc:2454] T 0552892f92ac466bab43afe5e17fd919 P f5d388a777fb40858d7ec7e12c1f2c39 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6512870b58747aeae6dd11f0bae6e8e in term 0.
I20260806 22:49:04.250839 12968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0552892f92ac466bab43afe5e17fd919" candidate_uuid: "f6512870b58747aeae6dd11f0bae6e8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60db83400a78493499d5a8ed3a7c7d1f" is_pre_election: true
I20260806 22:49:04.250978 13059 leader_election.cc:302] T 0552892f92ac466bab43afe5e17fd919 P f6512870b58747aeae6dd11f0bae6e8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f5d388a777fb40858d7ec7e12c1f2c39, f6512870b58747aeae6dd11f0bae6e8e; no voters:
I20260806 22:49:04.251067 12968 raft_consensus.cc:2454] T 0552892f92ac466bab43afe5e17fd919 P 60db83400a78493499d5a8ed3a7c7d1f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6512870b58747aeae6dd11f0bae6e8e in term 0.
I20260806 22:49:04.251199 13433 raft_consensus.cc:2781] T 0552892f92ac466bab43afe5e17fd919 P f6512870b58747aeae6dd11f0bae6e8e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:49:04.251266 13433 raft_consensus.cc:482] T 0552892f92ac466bab43afe5e17fd919 P f6512870b58747aeae6dd11f0bae6e8e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:49:04.251287 13433 raft_consensus.cc:3037] T 0552892f92ac466bab43afe5e17fd919 P f6512870b58747aeae6dd11f0bae6e8e [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:04.252332 13433 raft_consensus.cc:504] T 0552892f92ac466bab43afe5e17fd919 P f6512870b58747aeae6dd11f0bae6e8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db83400a78493499d5a8ed3a7c7d1f" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 36131 } } peers { permanent_uuid: "f6512870b58747aeae6dd11f0bae6e8e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45259 } } peers { permanent_uuid: "f5d388a777fb40858d7ec7e12c1f2c39" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 41125 } }
I20260806 22:49:04.252516 13433 leader_election.cc:288] T 0552892f92ac466bab43afe5e17fd919 P f6512870b58747aeae6dd11f0bae6e8e [CANDIDATE]: Term 1 election: Requested vote from peers 60db83400a78493499d5a8ed3a7c7d1f (127.18.45.193:36131), f5d388a777fb40858d7ec7e12c1f2c39 (127.18.45.195:41125)
I20260806 22:49:04.252730 13340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0552892f92ac466bab43afe5e17fd919" candidate_uuid: "f6512870b58747aeae6dd11f0bae6e8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5d388a777fb40858d7ec7e12c1f2c39"
I20260806 22:49:04.252770 12968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0552892f92ac466bab43afe5e17fd919" candidate_uuid: "f6512870b58747aeae6dd11f0bae6e8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60db83400a78493499d5a8ed3a7c7d1f"
I20260806 22:49:04.252907 12968 raft_consensus.cc:3037] T 0552892f92ac466bab43afe5e17fd919 P 60db83400a78493499d5a8ed3a7c7d1f [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:04.252833 13340 raft_consensus.cc:3037] T 0552892f92ac466bab43afe5e17fd919 P f5d388a777fb40858d7ec7e12c1f2c39 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:04.253885 13340 raft_consensus.cc:2454] T 0552892f92ac466bab43afe5e17fd919 P f5d388a777fb40858d7ec7e12c1f2c39 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6512870b58747aeae6dd11f0bae6e8e in term 1.
I20260806 22:49:04.253887 12968 raft_consensus.cc:2454] T 0552892f92ac466bab43afe5e17fd919 P 60db83400a78493499d5a8ed3a7c7d1f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6512870b58747aeae6dd11f0bae6e8e in term 1.
I20260806 22:49:04.254073 13059 leader_election.cc:302] T 0552892f92ac466bab43afe5e17fd919 P f6512870b58747aeae6dd11f0bae6e8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f5d388a777fb40858d7ec7e12c1f2c39, f6512870b58747aeae6dd11f0bae6e8e; no voters:
I20260806 22:49:04.254222 13433 raft_consensus.cc:2781] T 0552892f92ac466bab43afe5e17fd919 P f6512870b58747aeae6dd11f0bae6e8e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:49:04.254480 13433 raft_consensus.cc:686] T 0552892f92ac466bab43afe5e17fd919 P f6512870b58747aeae6dd11f0bae6e8e [term 1 LEADER]: Becoming Leader. State: Replica: f6512870b58747aeae6dd11f0bae6e8e, State: Running, Role: LEADER
I20260806 22:49:04.254617 13433 consensus_queue.cc:237] T 0552892f92ac466bab43afe5e17fd919 P f6512870b58747aeae6dd11f0bae6e8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db83400a78493499d5a8ed3a7c7d1f" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 36131 } } peers { permanent_uuid: "f6512870b58747aeae6dd11f0bae6e8e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45259 } } peers { permanent_uuid: "f5d388a777fb40858d7ec7e12c1f2c39" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 41125 } }
I20260806 22:49:04.255824 12775 catalog_manager.cc:5495] T 0552892f92ac466bab43afe5e17fd919 P f6512870b58747aeae6dd11f0bae6e8e reported cstate change: term changed from 0 to 1, leader changed from <none> to f6512870b58747aeae6dd11f0bae6e8e (127.18.45.194). New cstate: current_term: 1 leader_uuid: "f6512870b58747aeae6dd11f0bae6e8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60db83400a78493499d5a8ed3a7c7d1f" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 36131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6512870b58747aeae6dd11f0bae6e8e" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45259 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5d388a777fb40858d7ec7e12c1f2c39" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 41125 } health_report { overall_health: UNKNOWN } } }
I20260806 22:49:04.279076 13424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:04.276329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37670 (local address 127.18.45.194:45259)
0806 22:49:04.276422 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:04.276424 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:04.276432 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:49:04.276480 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:04.276481 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:04.276508 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:04.276591 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:49:04.276784 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:04.277318 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:04.278143 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:04.278363 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:04.278443 (+ 80us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:04.278630 (+ 187us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:49:04.278739 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:49:04.278861 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:04.278893 (+ 32us) server_negotiation.cc:299] Negotiation successful
0806 22:49:04.278908 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 22:49:04.281611 12968 raft_consensus.cc:1261] T 0552892f92ac466bab43afe5e17fd919 P 60db83400a78493499d5a8ed3a7c7d1f [term 1 FOLLOWER]: Refusing update from remote peer f6512870b58747aeae6dd11f0bae6e8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:49:04.281610 13340 raft_consensus.cc:1261] T 0552892f92ac466bab43afe5e17fd919 P f5d388a777fb40858d7ec7e12c1f2c39 [term 1 FOLLOWER]: Refusing update from remote peer f6512870b58747aeae6dd11f0bae6e8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:49:04.282083 13433 consensus_queue.cc:1035] T 0552892f92ac466bab43afe5e17fd919 P f6512870b58747aeae6dd11f0bae6e8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "60db83400a78493499d5a8ed3a7c7d1f" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 36131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:49:04.282271 13440 consensus_queue.cc:1035] T 0552892f92ac466bab43afe5e17fd919 P f6512870b58747aeae6dd11f0bae6e8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5d388a777fb40858d7ec7e12c1f2c39" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 41125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:49:04.285359 13446 mvcc.cc:205] Tried to move back new op lower bound from 7315687605373992960 to 7315687605267738624. Current Snapshot: MvccSnapshot[applied={T|T < 7315687605373992960}]
I20260806 22:49:04.285776 13444 mvcc.cc:205] Tried to move back new op lower bound from 7315687605373992960 to 7315687605267738624. Current Snapshot: MvccSnapshot[applied={T|T < 7315687605373992960}]
I20260806 22:49:04.288944 12774 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55500:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:49:04.289075 12774 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55500:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:49:04.290073 12774 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6fceb0fa8170492abe9bcf4dfb93f92f: Sending DeleteTablet for 3 replicas of tablet 0552892f92ac466bab43afe5e17fd919
I20260806 22:49:04.290457 12941 tablet_service.cc:1514] Processing DeleteTablet for tablet 0552892f92ac466bab43afe5e17fd919 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:49:04 UTC) from {username='slave'} at 127.0.0.1:52096
I20260806 22:49:04.290485 13129 tablet_service.cc:1514] Processing DeleteTablet for tablet 0552892f92ac466bab43afe5e17fd919 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:49:04 UTC) from {username='slave'} at 127.0.0.1:37664
I20260806 22:49:04.290596 13311 tablet_service.cc:1514] Processing DeleteTablet for tablet 0552892f92ac466bab43afe5e17fd919 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:49:04 UTC) from {username='slave'} at 127.0.0.1:44366
I20260806 22:49:04.290846 13459 tablet_replica.cc:330] T 0552892f92ac466bab43afe5e17fd919 P f5d388a777fb40858d7ec7e12c1f2c39: stopping tablet replica
I20260806 22:49:04.290848 13457 tablet_replica.cc:330] T 0552892f92ac466bab43afe5e17fd919 P 60db83400a78493499d5a8ed3a7c7d1f: stopping tablet replica
I20260806 22:49:04.290966 13457 raft_consensus.cc:2229] T 0552892f92ac466bab43afe5e17fd919 P 60db83400a78493499d5a8ed3a7c7d1f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:49:04.290966 13459 raft_consensus.cc:2229] T 0552892f92ac466bab43afe5e17fd919 P f5d388a777fb40858d7ec7e12c1f2c39 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:49:04.291121 13459 raft_consensus.cc:2258] T 0552892f92ac466bab43afe5e17fd919 P f5d388a777fb40858d7ec7e12c1f2c39 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:49:04.291450 13457 raft_consensus.cc:2258] T 0552892f92ac466bab43afe5e17fd919 P 60db83400a78493499d5a8ed3a7c7d1f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:49:04.291463 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 12857
I20260806 22:49:04.291666 13459 ts_tablet_manager.cc:1905] T 0552892f92ac466bab43afe5e17fd919 P f5d388a777fb40858d7ec7e12c1f2c39: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:49:04.294550 13459 ts_tablet_manager.cc:1918] T 0552892f92ac466bab43afe5e17fd919 P f5d388a777fb40858d7ec7e12c1f2c39: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:49:04.294636 13459 log.cc:1196] T 0552892f92ac466bab43afe5e17fd919 P f5d388a777fb40858d7ec7e12c1f2c39: Deleting WAL directory at /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056275803658-18615-0/minicluster-data/ts-2/wal/wals/0552892f92ac466bab43afe5e17fd919
I20260806 22:49:04.294915 13459 ts_tablet_manager.cc:1939] T 0552892f92ac466bab43afe5e17fd919 P f5d388a777fb40858d7ec7e12c1f2c39: Deleting consensus metadata
I20260806 22:49:04.295424 12758 catalog_manager.cc:4841] TS f5d388a777fb40858d7ec7e12c1f2c39 (127.18.45.195:41125): tablet 0552892f92ac466bab43afe5e17fd919 (table test-table [id=6deefc68b1c447fa91ce9a693a8056e1]) successfully deleted
I20260806 22:49:04.303175 13458 tablet_replica.cc:330] T 0552892f92ac466bab43afe5e17fd919 P f6512870b58747aeae6dd11f0bae6e8e: stopping tablet replica
I20260806 22:49:04.303359 13458 raft_consensus.cc:2229] T 0552892f92ac466bab43afe5e17fd919 P f6512870b58747aeae6dd11f0bae6e8e [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:49:04.303566 13458 raft_consensus.cc:2258] T 0552892f92ac466bab43afe5e17fd919 P f6512870b58747aeae6dd11f0bae6e8e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:49:04.304236 13458 ts_tablet_manager.cc:1905] T 0552892f92ac466bab43afe5e17fd919 P f6512870b58747aeae6dd11f0bae6e8e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:49:04.306294 13458 ts_tablet_manager.cc:1918] T 0552892f92ac466bab43afe5e17fd919 P f6512870b58747aeae6dd11f0bae6e8e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:49:04.306387 13458 log.cc:1196] T 0552892f92ac466bab43afe5e17fd919 P f6512870b58747aeae6dd11f0bae6e8e: Deleting WAL directory at /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786056275803658-18615-0/minicluster-data/ts-1/wal/wals/0552892f92ac466bab43afe5e17fd919
I20260806 22:49:04.306694 13458 ts_tablet_manager.cc:1939] T 0552892f92ac466bab43afe5e17fd919 P f6512870b58747aeae6dd11f0bae6e8e: Deleting consensus metadata
I20260806 22:49:04.307348 12755 catalog_manager.cc:4841] TS f6512870b58747aeae6dd11f0bae6e8e (127.18.45.194:45259): tablet 0552892f92ac466bab43afe5e17fd919 (table test-table [id=6deefc68b1c447fa91ce9a693a8056e1]) successfully deleted
W20260806 22:49:04.308072 12756 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.45.193:36131 (error 108)
I20260806 22:49:04.308769 13422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:04.308591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.193:36131 (local address 127.0.0.1:52108)
0806 22:49:04.308666 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:04.308722 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.193:36131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260806 22:49:04.308854 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 13035
W20260806 22:49:04.314967 12756 catalog_manager.cc:4584] TS 60db83400a78493499d5a8ed3a7c7d1f (127.18.45.193:36131): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0552892f92ac466bab43afe5e17fd919: Network error: Client connection negotiation failed: client connection to 127.18.45.193:36131: connect: Connection refused (error 111)
I20260806 22:49:04.329360 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 13221
I20260806 22:49:04.342069 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 12734
2026-08-06T22:49:04Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (904 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:49:04 dist-test-slave-h3ft krb5kdc[13464](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:49:04 dist-test-slave-h3ft krb5kdc[13464](info): set up 2 sockets
Aug 06 22:49:04 dist-test-slave-h3ft krb5kdc[13464](info): commencing operation
krb5kdc: starting...
W20260806 22:49:05.667225 18615 mini_kdc.cc:121] Time spent starting KDC: real 1.296s user 0.002s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Aug 06 22:49:05 dist-test-slave-h3ft krb5kdc[13464](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056545, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:49:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:49:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.45.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab.
Entry for principal kudu/127.18.45.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab.
I20260806 22:49:05.832224 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:42109
--webserver_interface=127.18.45.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:34901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.45.254:42109
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:49:05.932073 13481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:05.932358 13481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:05.932385 13481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:05.933723 13481 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:49:05.933775 13481 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:05.933790 13481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:05.933804 13481 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:49:05.933815 13481 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:49:05.935650 13481 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:34901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.45.254:42109
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:42109
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.45.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13481
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:49:05.935982 13481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:05.936285 13481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:05.942595 13491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:05.942610 13490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:05.942660 13493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:05.943080 13481 server_base.cc:1034] running on GCE node
I20260806 22:49:05.943512 13481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:05.943926 13481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:05.945109 13481 hybrid_clock.cc:648] HybridClock initialized: now 1786056545945092 us; error 69 us; skew 500 ppm
Aug 06 22:49:05 dist-test-slave-h3ft krb5kdc[13464](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056545, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:05.948388 13481 init.cc:377] Logged in from keytab as kudu/127.18.45.254@KRBTEST.COM (short username kudu)
I20260806 22:49:05.948992 13481 webserver.cc:460] Webserver started at http://127.18.45.254:39751/ using document root <none> and password file <none>
I20260806 22:49:05.949303 13481 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:05.949384 13481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:05.949555 13481 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:49:05.950857 13481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/minicluster-data/master-0/data/instance:
uuid: "eab799275ad245db88c37299e74cc761"
format_stamp: "Formatted at 2026-08-06 22:49:05 on dist-test-slave-h3ft"
server_key: "125b2af004dbef6ba40c2c566c8bdd28"
server_key_iv: "5db418323fd7f77bf532481bd4dff7ca"
server_key_version: "encryptionkey@0"
I20260806 22:49:05.951354 13481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/minicluster-data/master-0/wal/instance:
uuid: "eab799275ad245db88c37299e74cc761"
format_stamp: "Formatted at 2026-08-06 22:49:05 on dist-test-slave-h3ft"
server_key: "125b2af004dbef6ba40c2c566c8bdd28"
server_key_iv: "5db418323fd7f77bf532481bd4dff7ca"
server_key_version: "encryptionkey@0"
I20260806 22:49:05.952981 13481 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:49:05.953998 13507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:05.954316 13481 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:49:05.954468 13481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/minicluster-data/master-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/minicluster-data/master-0/wal
uuid: "eab799275ad245db88c37299e74cc761"
format_stamp: "Formatted at 2026-08-06 22:49:05 on dist-test-slave-h3ft"
server_key: "125b2af004dbef6ba40c2c566c8bdd28"
server_key_iv: "5db418323fd7f77bf532481bd4dff7ca"
server_key_version: "encryptionkey@0"
I20260806 22:49:05.954591 13481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:05.968887 13481 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:05.972024 13481 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:05.972237 13481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:05.977699 13481 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.254:42109
I20260806 22:49:05.977795 13572 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.254:42109 every 8 connection(s)
I20260806 22:49:05.978222 13481 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
I20260806 22:49:05.979439 13575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:05.980453 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 13481
I20260806 22:49:05.980616 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/minicluster-data/master-0/wal/instance
I20260806 22:49:05.980857 18615 external_mini_cluster.cc:1428] Setting key 387100da2ef1c5418e26067c46a1f702
I20260806 22:49:05.983479 13575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eab799275ad245db88c37299e74cc761: Bootstrap starting.
I20260806 22:49:05.984381 13575 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eab799275ad245db88c37299e74cc761: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:05.984762 13575 log.cc:824] T 00000000000000000000000000000000 P eab799275ad245db88c37299e74cc761: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:05.985595 13575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eab799275ad245db88c37299e74cc761: No bootstrap required, opened a new log
I20260806 22:49:05.987950 13575 raft_consensus.cc:348] T 00000000000000000000000000000000 P eab799275ad245db88c37299e74cc761 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eab799275ad245db88c37299e74cc761" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 42109 } }
I20260806 22:49:05.988143 13575 raft_consensus.cc:374] T 00000000000000000000000000000000 P eab799275ad245db88c37299e74cc761 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:05.988219 13575 raft_consensus.cc:729] T 00000000000000000000000000000000 P eab799275ad245db88c37299e74cc761 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eab799275ad245db88c37299e74cc761, State: Initialized, Role: FOLLOWER
I20260806 22:49:05.988391 13575 consensus_queue.cc:260] T 00000000000000000000000000000000 P eab799275ad245db88c37299e74cc761 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eab799275ad245db88c37299e74cc761" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 42109 } }
I20260806 22:49:05.988489 13575 raft_consensus.cc:388] T 00000000000000000000000000000000 P eab799275ad245db88c37299e74cc761 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:49:05.988543 13575 raft_consensus.cc:482] T 00000000000000000000000000000000 P eab799275ad245db88c37299e74cc761 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:49:05.988597 13575 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eab799275ad245db88c37299e74cc761 [term 0 FOLLOWER]: Advancing to term 1
Aug 06 22:49:05 dist-test-slave-h3ft krb5kdc[13464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056545, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:49:05.989313 13575 raft_consensus.cc:504] T 00000000000000000000000000000000 P eab799275ad245db88c37299e74cc761 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eab799275ad245db88c37299e74cc761" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 42109 } }
I20260806 22:49:05.989457 13575 leader_election.cc:302] T 00000000000000000000000000000000 P eab799275ad245db88c37299e74cc761 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eab799275ad245db88c37299e74cc761; no voters:
I20260806 22:49:05.989675 13575 leader_election.cc:288] T 00000000000000000000000000000000 P eab799275ad245db88c37299e74cc761 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:49:05.989753 13580 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eab799275ad245db88c37299e74cc761 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:49:05.990048 13575 sys_catalog.cc:564] T 00000000000000000000000000000000 P eab799275ad245db88c37299e74cc761 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:49:05.990080 13580 raft_consensus.cc:686] T 00000000000000000000000000000000 P eab799275ad245db88c37299e74cc761 [term 1 LEADER]: Becoming Leader. State: Replica: eab799275ad245db88c37299e74cc761, State: Running, Role: LEADER
I20260806 22:49:05.990193 13580 consensus_queue.cc:237] T 00000000000000000000000000000000 P eab799275ad245db88c37299e74cc761 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eab799275ad245db88c37299e74cc761" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 42109 } }
I20260806 22:49:05.990638 13582 sys_catalog.cc:455] T 00000000000000000000000000000000 P eab799275ad245db88c37299e74cc761 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eab799275ad245db88c37299e74cc761" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eab799275ad245db88c37299e74cc761" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 42109 } } }
I20260806 22:49:05.990751 13582 sys_catalog.cc:458] T 00000000000000000000000000000000 P eab799275ad245db88c37299e74cc761 [sys.catalog]: This master's current role is: LEADER
I20260806 22:49:05.990811 13583 sys_catalog.cc:455] T 00000000000000000000000000000000 P eab799275ad245db88c37299e74cc761 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eab799275ad245db88c37299e74cc761. Latest consensus state: current_term: 1 leader_uuid: "eab799275ad245db88c37299e74cc761" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eab799275ad245db88c37299e74cc761" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 42109 } } }
I20260806 22:49:05.990928 13583 sys_catalog.cc:458] T 00000000000000000000000000000000 P eab799275ad245db88c37299e74cc761 [sys.catalog]: This master's current role is: LEADER
I20260806 22:49:05.991164 13593 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:49:05.991885 13593 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:49:05.994136 13578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:05.981520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58286 (local address 127.18.45.254:42109)
0806 22:49:05.981708 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:05.981719 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:05.981756 (+ 37us) server_negotiation.cc:407] Connection header received
0806 22:49:05.982412 (+ 656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:05.982423 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:05.982502 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:05.982683 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:05.983480 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:05.984986 (+ 1506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:05.985906 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:05.986387 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:05.986536 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:05.989549 (+ 3013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:05.989556 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:05.989556 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:05.989571 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:05.992130 (+ 2559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:05.992619 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:05.992622 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:05.992623 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:05.992737 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:05.993002 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:05.993003 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:05.993003 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:05.993405 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:05.993584 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:05.993715 (+ 131us) server_negotiation.cc:299] Negotiation successful
0806 22:49:05.993883 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":86,"threads_started":1}
I20260806 22:49:05.994411 13593 catalog_manager.cc:1347] Generated new cluster ID: f1c70ca343f743de89b66b0cfe3b30cd
I20260806 22:49:05.994472 13593 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:49:06.011582 13593 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:49:06.012333 13593 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:49:06.018975 13593 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eab799275ad245db88c37299e74cc761: Generated new TSK 0
I20260806 22:49:06.019291 13593 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.45.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab.
Entry for principal kudu/127.18.45.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab.
I20260806 22:49:06.068651 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.193:0
--local_ip_for_outbound_sockets=127.18.45.193
--webserver_interface=127.18.45.193
--webserver_port=0
--tserver_master_addrs=127.18.45.254:42109
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab
--principal=kudu/127.18.45.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:34901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:49:06.169093 13609 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:06.169448 13609 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:06.169497 13609 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:06.170864 13609 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:06.170953 13609 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:06.171003 13609 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.193
I20260806 22:49:06.172757 13609 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:34901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.193.keytab
--principal=kudu/127.18.45.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.45.193
--webserver_port=0
--tserver_master_addrs=127.18.45.254:42109
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13609
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.193
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:49:06.173103 13609 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:06.173424 13609 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:06.180191 13616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:06.180191 13617 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:06.180229 13609 server_base.cc:1034] running on GCE node
W20260806 22:49:06.180191 13619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:06.180636 13609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:06.181032 13609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:06.182263 13609 hybrid_clock.cc:648] HybridClock initialized: now 1786056546182245 us; error 30 us; skew 500 ppm
Aug 06 22:49:06 dist-test-slave-h3ft krb5kdc[13464](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056546, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:06.186681 13609 init.cc:377] Logged in from keytab as kudu/127.18.45.193@KRBTEST.COM (short username kudu)
I20260806 22:49:06.187367 13609 webserver.cc:460] Webserver started at http://127.18.45.193:41507/ using document root <none> and password file <none>
I20260806 22:49:06.187698 13609 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:06.187779 13609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:06.187958 13609 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:49:06.189345 13609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/minicluster-data/ts-0/data/instance:
uuid: "72749adaae664ad7a1908ffca648d8b8"
format_stamp: "Formatted at 2026-08-06 22:49:06 on dist-test-slave-h3ft"
server_key: "ad3253e37a2a5e2a01bf77f178187c42"
server_key_iv: "94ea62ed9c24698adb0ad2d3e0bbea77"
server_key_version: "encryptionkey@0"
I20260806 22:49:06.189800 13609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/minicluster-data/ts-0/wal/instance:
uuid: "72749adaae664ad7a1908ffca648d8b8"
format_stamp: "Formatted at 2026-08-06 22:49:06 on dist-test-slave-h3ft"
server_key: "ad3253e37a2a5e2a01bf77f178187c42"
server_key_iv: "94ea62ed9c24698adb0ad2d3e0bbea77"
server_key_version: "encryptionkey@0"
I20260806 22:49:06.191725 13609 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.003s
I20260806 22:49:06.192870 13626 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:06.193207 13609 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:49:06.193328 13609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/minicluster-data/ts-0/wal
uuid: "72749adaae664ad7a1908ffca648d8b8"
format_stamp: "Formatted at 2026-08-06 22:49:06 on dist-test-slave-h3ft"
server_key: "ad3253e37a2a5e2a01bf77f178187c42"
server_key_iv: "94ea62ed9c24698adb0ad2d3e0bbea77"
server_key_version: "encryptionkey@0"
I20260806 22:49:06.193428 13609 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:06.203828 13609 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:06.206215 13609 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:06.206409 13609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:06.206733 13609 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:49:06.207191 13609 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:49:06.207254 13609 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:06.207326 13609 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:49:06.207357 13609 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:06.214676 13609 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.193:32859
I20260806 22:49:06.214771 13783 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.193:32859 every 8 connection(s)
I20260806 22:49:06.215219 13609 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
I20260806 22:49:06.217300 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 13609
I20260806 22:49:06.217463 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/minicluster-data/ts-0/wal/instance
I20260806 22:49:06.217751 18615 external_mini_cluster.cc:1428] Setting key 871879c9500074002b955ddb52325668
Aug 06 22:49:06 dist-test-slave-h3ft krb5kdc[13464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056546, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.193@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:49:06.229039 13578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:06.216331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:38707 (local address 127.18.45.254:42109)
0806 22:49:06.216403 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:06.216405 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:06.216691 (+ 286us) server_negotiation.cc:407] Connection header received
0806 22:49:06.217555 (+ 864us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:06.217557 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:06.217573 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:06.217652 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:06.219100 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:06.219669 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:06.221383 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:06.221768 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:06.221951 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:06.224915 (+ 2964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:06.224918 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:06.224918 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:06.224930 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:06.227120 (+ 2190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:06.227675 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:06.227678 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:06.227679 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:06.227770 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:06.228119 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:06.228121 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:06.228121 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:06.228330 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:06.228420 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:06.228768 (+ 348us) server_negotiation.cc:299] Negotiation successful
0806 22:49:06.228911 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 22:49:06.229189 13786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:06.216289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:42109 (local address 127.18.45.193:38707)
0806 22:49:06.216500 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:06.216553 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:06.217362 (+ 809us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:06.217749 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:06.217754 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:06.218548 (+ 794us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:06.218930 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:06.218939 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:06.219790 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:06.219791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:06.221253 (+ 1462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:06.221259 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:06.221957 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:06.221958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:06.222101 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:06.222875 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:49:06.222886 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:06.224793 (+ 1907us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:06.227248 (+ 2455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:06.227258 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:06.227260 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:06.227571 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:06.227881 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:06.227883 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:06.227883 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:06.228033 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:06.228433 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:06.228443 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:06.228606 (+ 163us) client_negotiation.cc:769] Sending connection context
0806 22:49:06.228710 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 22:49:06.228873 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":114,"threads_started":1}
I20260806 22:49:06.229981 13784 heartbeater.cc:347] Connected to a master server at 127.18.45.254:42109
I20260806 22:49:06.230096 13784 heartbeater.cc:464] Registering TS with master...
I20260806 22:49:06.230351 13784 heartbeater.cc:511] Master 127.18.45.254:42109 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.18.45.194@KRBTEST.COM; defaulting to no policy
I20260806 22:49:06.231202 13527 ts_manager.cc:194] Registered new tserver with Master: 72749adaae664ad7a1908ffca648d8b8 (127.18.45.193:32859)
I20260806 22:49:06.232024 13527 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.45.193@KRBTEST.COM'} at 127.18.45.193:38707
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab.
Entry for principal kudu/127.18.45.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab.
I20260806 22:49:06.255442 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.194:0
--local_ip_for_outbound_sockets=127.18.45.194
--webserver_interface=127.18.45.194
--webserver_port=0
--tserver_master_addrs=127.18.45.254:42109
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab
--principal=kudu/127.18.45.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:34901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:49:06.355073 13789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:06.355355 13789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:06.355384 13789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:06.356563 13789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:06.356609 13789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:06.356638 13789 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.194
I20260806 22:49:06.358469 13789 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:34901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.194.keytab
--principal=kudu/127.18.45.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.45.194
--webserver_port=0
--tserver_master_addrs=127.18.45.254:42109
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13789
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.194
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:49:06.358839 13789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:06.359189 13789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 22:49:06.365640 13789 server_base.cc:1034] running on GCE node
W20260806 22:49:06.365577 13798 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:06.365569 13799 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:06.365577 13805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:06.366161 13789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:06.366461 13789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:06.367648 13789 hybrid_clock.cc:648] HybridClock initialized: now 1786056546367636 us; error 32 us; skew 500 ppm
Aug 06 22:49:06 dist-test-slave-h3ft krb5kdc[13464](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056546, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:06.371085 13789 init.cc:377] Logged in from keytab as kudu/127.18.45.194@KRBTEST.COM (short username kudu)
I20260806 22:49:06.371739 13789 webserver.cc:460] Webserver started at http://127.18.45.194:34615/ using document root <none> and password file <none>
I20260806 22:49:06.372061 13789 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:06.372146 13789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:06.372332 13789 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:49:06.373833 13789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/minicluster-data/ts-1/data/instance:
uuid: "cbab717dc326471ead3e4359e63c7c63"
format_stamp: "Formatted at 2026-08-06 22:49:06 on dist-test-slave-h3ft"
server_key: "709ba1126f766993bc5ded2ea4835064"
server_key_iv: "cb4ec95379715bf4edc3ea004dbb8d4b"
server_key_version: "encryptionkey@0"
I20260806 22:49:06.374361 13789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/minicluster-data/ts-1/wal/instance:
uuid: "cbab717dc326471ead3e4359e63c7c63"
format_stamp: "Formatted at 2026-08-06 22:49:06 on dist-test-slave-h3ft"
server_key: "709ba1126f766993bc5ded2ea4835064"
server_key_iv: "cb4ec95379715bf4edc3ea004dbb8d4b"
server_key_version: "encryptionkey@0"
I20260806 22:49:06.376171 13789 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260806 22:49:06.377182 13814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:06.377545 13789 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:49:06.377676 13789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/minicluster-data/ts-1/wal
uuid: "cbab717dc326471ead3e4359e63c7c63"
format_stamp: "Formatted at 2026-08-06 22:49:06 on dist-test-slave-h3ft"
server_key: "709ba1126f766993bc5ded2ea4835064"
server_key_iv: "cb4ec95379715bf4edc3ea004dbb8d4b"
server_key_version: "encryptionkey@0"
I20260806 22:49:06.377765 13789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:06.404532 13789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:06.407333 13789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:06.407567 13789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:06.408003 13789 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:49:06.408437 13789 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:49:06.408501 13789 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:06.408543 13789 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:49:06.408592 13789 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:06.416501 13789 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.194:45865
I20260806 22:49:06.416574 13976 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.194:45865 every 8 connection(s)
I20260806 22:49:06.417029 13789 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
I20260806 22:49:06.424149 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 13789
I20260806 22:49:06.424312 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/minicluster-data/ts-1/wal/instance
I20260806 22:49:06.424594 18615 external_mini_cluster.cc:1428] Setting key 5ab18b38455c43b99677c7048ea97a4e
Aug 06 22:49:06 dist-test-slave-h3ft krb5kdc[13464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056546, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.194@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:49:06.428877 13578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:06.418115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:37331 (local address 127.18.45.254:42109)
0806 22:49:06.418216 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:06.418219 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:06.418438 (+ 219us) server_negotiation.cc:407] Connection header received
0806 22:49:06.419103 (+ 665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:06.419104 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:06.419119 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:06.419190 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:06.420392 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:06.421030 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:06.422214 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:06.422453 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:06.422525 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:06.425310 (+ 2785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:06.425312 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:06.425312 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:06.425321 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:06.427053 (+ 1732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:06.427642 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:06.427644 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:06.427644 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:06.427707 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:06.428057 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:06.428058 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:06.428058 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:06.428246 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:06.428325 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:06.428625 (+ 300us) server_negotiation.cc:299] Negotiation successful
0806 22:49:06.428732 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 22:49:06.429004 13981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:06.418016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:42109 (local address 127.18.45.194:37331)
0806 22:49:06.418248 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:06.418300 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:06.418926 (+ 626us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:06.419285 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:06.419290 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:06.420025 (+ 735us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:06.420282 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:06.420287 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:06.421127 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:06.421128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:06.422057 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:06.422060 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:06.422545 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:06.422546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:06.422674 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:06.423300 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:49:06.423310 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:06.425213 (+ 1903us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:06.427140 (+ 1927us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:06.427150 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:06.427153 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:06.427553 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:06.427777 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:06.427778 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:06.427778 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:06.427975 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:06.428320 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:06.428327 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:06.428484 (+ 157us) client_negotiation.cc:769] Sending connection context
0806 22:49:06.428583 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 22:49:06.428742 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":115,"threads_started":1}
I20260806 22:49:06.429668 13977 heartbeater.cc:347] Connected to a master server at 127.18.45.254:42109
I20260806 22:49:06.429805 13977 heartbeater.cc:464] Registering TS with master...
I20260806 22:49:06.430071 13977 heartbeater.cc:511] Master 127.18.45.254:42109 requested a full tablet report, sending...
I20260806 22:49:06.430593 13527 ts_manager.cc:194] Registered new tserver with Master: cbab717dc326471ead3e4359e63c7c63 (127.18.45.194:45865)
I20260806 22:49:06.431222 13527 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.45.194@KRBTEST.COM'} at 127.18.45.194:37331
WARNING: no policy specified for kudu/127.18.45.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab.
Entry for principal kudu/127.18.45.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab.
I20260806 22:49:06.459298 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.195:0
--local_ip_for_outbound_sockets=127.18.45.195
--webserver_interface=127.18.45.195
--webserver_port=0
--tserver_master_addrs=127.18.45.254:42109
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab
--principal=kudu/127.18.45.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:34901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:49:06.558903 13986 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:06.559264 13986 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:06.559297 13986 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:06.560612 13986 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:06.560660 13986 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:06.560689 13986 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.195
I20260806 22:49:06.562321 13986 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:34901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.195.keytab
--principal=kudu/127.18.45.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.45.195
--webserver_port=0
--tserver_master_addrs=127.18.45.254:42109
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13986
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.195
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:49:06.562619 13986 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:06.562913 13986 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:06.568830 14003 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:06.568988 13986 server_base.cc:1034] running on GCE node
W20260806 22:49:06.568846 13995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:06.569018 13992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:06.569319 13986 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:06.569615 13986 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:06.570815 13986 hybrid_clock.cc:648] HybridClock initialized: now 1786056546570792 us; error 43 us; skew 500 ppm
Aug 06 22:49:06 dist-test-slave-h3ft krb5kdc[13464](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056546, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:06.574783 13986 init.cc:377] Logged in from keytab as kudu/127.18.45.195@KRBTEST.COM (short username kudu)
I20260806 22:49:06.575414 13986 webserver.cc:460] Webserver started at http://127.18.45.195:33507/ using document root <none> and password file <none>
I20260806 22:49:06.575683 13986 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:06.575759 13986 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:06.575953 13986 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:49:06.577255 13986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/minicluster-data/ts-2/data/instance:
uuid: "5864b7cd1ef347cb8fb7c63639e5f808"
format_stamp: "Formatted at 2026-08-06 22:49:06 on dist-test-slave-h3ft"
server_key: "6efbaa126084411302a7ca900243034d"
server_key_iv: "23638eb76b27a1f7a32f5ea5e1068d37"
server_key_version: "encryptionkey@0"
I20260806 22:49:06.577682 13986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/minicluster-data/ts-2/wal/instance:
uuid: "5864b7cd1ef347cb8fb7c63639e5f808"
format_stamp: "Formatted at 2026-08-06 22:49:06 on dist-test-slave-h3ft"
server_key: "6efbaa126084411302a7ca900243034d"
server_key_iv: "23638eb76b27a1f7a32f5ea5e1068d37"
server_key_version: "encryptionkey@0"
I20260806 22:49:06.579329 13986 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:49:06.580276 14016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:06.580515 13986 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:49:06.580639 13986 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/minicluster-data/ts-2/wal
uuid: "5864b7cd1ef347cb8fb7c63639e5f808"
format_stamp: "Formatted at 2026-08-06 22:49:06 on dist-test-slave-h3ft"
server_key: "6efbaa126084411302a7ca900243034d"
server_key_iv: "23638eb76b27a1f7a32f5ea5e1068d37"
server_key_version: "encryptionkey@0"
I20260806 22:49:06.580724 13986 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:06.599803 13986 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:06.602344 13986 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:06.602551 13986 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:06.602862 13986 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:49:06.603312 13986 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:49:06.603374 13986 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:06.603425 13986 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:49:06.603453 13986 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:06.610523 13986 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.195:33443
I20260806 22:49:06.610602 14175 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.195:33443 every 8 connection(s)
I20260806 22:49:06.610966 13986 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
I20260806 22:49:06.617268 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 13986
I20260806 22:49:06.617372 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/minicluster-data/ts-2/wal/instance
I20260806 22:49:06.617591 18615 external_mini_cluster.cc:1428] Setting key 44d180384aae6b39288de0ba28692967
Aug 06 22:49:06 dist-test-slave-h3ft krb5kdc[13464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056546, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.195@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:49:06.621816 13578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:06.611873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.195:54825 (local address 127.18.45.254:42109)
0806 22:49:06.611983 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:06.611986 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:06.612154 (+ 168us) server_negotiation.cc:407] Connection header received
0806 22:49:06.612771 (+ 617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:06.612772 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:06.612785 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:06.612837 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:06.613960 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:06.614524 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:06.615663 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:06.615871 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:06.615931 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:06.618879 (+ 2948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:06.618880 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:06.618880 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:06.618888 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:06.620523 (+ 1635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:06.620939 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:06.620940 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:06.620940 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:06.621004 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:06.621242 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:06.621243 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:06.621243 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:06.621357 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:06.621414 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:06.621665 (+ 251us) server_negotiation.cc:299] Negotiation successful
0806 22:49:06.621738 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 22:49:06.622062 14180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:06.611807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:42109 (local address 127.18.45.195:54825)
0806 22:49:06.611997 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:06.612048 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:06.612669 (+ 621us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:06.612951 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:06.612955 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:06.613619 (+ 664us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:06.613857 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:06.613863 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:06.614627 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:06.614628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:06.615572 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:06.615576 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:06.615967 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:06.615968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:06.616096 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:06.616753 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:49:06.616762 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:06.618796 (+ 2034us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:06.620595 (+ 1799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:06.620602 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:06.620604 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:06.620839 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:06.621072 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:06.621072 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:06.621073 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:06.621184 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:06.621400 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:06.621403 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:06.621546 (+ 143us) client_negotiation.cc:769] Sending connection context
0806 22:49:06.621633 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 22:49:06.621772 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":99,"threads_started":1}
I20260806 22:49:06.622561 14176 heartbeater.cc:347] Connected to a master server at 127.18.45.254:42109
I20260806 22:49:06.622682 14176 heartbeater.cc:464] Registering TS with master...
I20260806 22:49:06.622900 14176 heartbeater.cc:511] Master 127.18.45.254:42109 requested a full tablet report, sending...
I20260806 22:49:06.623384 13527 ts_manager.cc:194] Registered new tserver with Master: 5864b7cd1ef347cb8fb7c63639e5f808 (127.18.45.195:33443)
I20260806 22:49:06.623801 13527 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.45.195@KRBTEST.COM'} at 127.18.45.195:54825
I20260806 22:49:06.629508 18615 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:49:06.637457 13578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:06.631226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58290 (local address 127.18.45.254:42109)
0806 22:49:06.631288 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:06.631290 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:06.631399 (+ 109us) server_negotiation.cc:407] Connection header received
0806 22:49:06.631442 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:06.631443 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:06.631453 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:06.631522 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:06.632105 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:06.632644 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:06.633366 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:06.633572 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:06.633627 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:06.634888 (+ 1261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:06.634889 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:06.634890 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:06.634897 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:06.636435 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:06.636789 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:06.636790 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:06.636791 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:06.636848 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:06.636979 (+ 131us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:06.636979 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:06.636980 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:06.637138 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:06.637218 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:06.637311 (+ 93us) server_negotiation.cc:299] Negotiation successful
0806 22:49:06.637380 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 22:49:06.639451 13527 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58290:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:49:06.641557 13527 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:49:06.649771 14195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:06.644962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.195:33443 (local address 127.0.0.1:33778)
0806 22:49:06.645887 (+ 925us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:06.645899 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:06.646010 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:06.646324 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:06.646324 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:06.646326 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:49:06.646378 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:06.646379 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:06.647433 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:06.647435 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:06.648545 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:06.648549 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:06.649385 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:06.649386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:06.649487 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:06.649505 (+ 18us) client_negotiation.cc:769] Sending connection context
0806 22:49:06.649590 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 22:49:06.649666 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":900,"mutex_wait_us":1,"thread_start_us":114,"threads_started":1}
I20260806 22:49:06.649786 14199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:06.645200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33778 (local address 127.18.45.195:33443)
0806 22:49:06.645489 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:06.645500 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:06.645979 (+ 479us) server_negotiation.cc:407] Connection header received
0806 22:49:06.646119 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:06.646123 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:06.646143 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:06.646208 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:49:06.646460 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:06.647311 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:06.648660 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:06.649285 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:06.649370 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:06.649456 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:06.649621 (+ 165us) server_negotiation.cc:299] Negotiation successful
0806 22:49:06.649684 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":127,"threads_started":1}
I20260806 22:49:06.650610 14196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:06.644942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.194:45865 (local address 127.0.0.1:57110)
0806 22:49:06.645217 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:06.645230 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:06.645316 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:06.645792 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:06.645797 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:06.645800 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:49:06.645918 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:06.645920 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:06.647696 (+ 1776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:06.647697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:06.649167 (+ 1470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:06.649171 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:06.650306 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:06.650307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:06.650403 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:06.650409 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 22:49:06.650456 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 22:49:06.650508 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"mutex_wait_us":129,"thread_start_us":110,"threads_started":1}
I20260806 22:49:06.650717 14198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:06.644939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.193:32859 (local address 127.0.0.1:54806)
0806 22:49:06.645168 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:06.645220 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:06.645316 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:06.646059 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:06.646060 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:06.646063 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:49:06.646133 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:06.646136 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:06.647697 (+ 1561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:06.647697 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:06.649169 (+ 1472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:06.649171 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:06.650446 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:06.650447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:06.650526 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:06.650531 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:49:06.650590 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 22:49:06.650641 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"mutex_wait_us":117,"thread_start_us":86,"threads_started":1}
I20260806 22:49:06.650828 14204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:06.645629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54806 (local address 127.18.45.193:32859)
0806 22:49:06.645871 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:06.645876 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:06.645891 (+ 15us) server_negotiation.cc:407] Connection header received
0806 22:49:06.645941 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:06.645943 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:06.645964 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:06.646066 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:49:06.646205 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:06.647535 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:06.649291 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:06.650328 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:06.650440 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:06.650509 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:06.650614 (+ 105us) server_negotiation.cc:299] Negotiation successful
0806 22:49:06.650693 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":129,"threads_started":1}
I20260806 22:49:06.651216 14202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:06.645311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57110 (local address 127.18.45.194:45865)
0806 22:49:06.645537 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:06.645542 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:06.645556 (+ 14us) server_negotiation.cc:407] Connection header received
0806 22:49:06.645635 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:06.645637 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:06.645658 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:06.646032 (+ 374us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:49:06.646154 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:06.647527 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:06.649296 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:06.650196 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:06.650900 (+ 704us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:06.650994 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:06.651052 (+ 58us) server_negotiation.cc:299] Negotiation successful
0806 22:49:06.651111 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":133,"threads_started":1}
I20260806 22:49:06.651192 14081 tablet_service.cc:1467] Processing CreateTablet for tablet 5a81579c8a2c466682938585880817f7 (DEFAULT_TABLE table=test-table [id=7dbf64438cf549a6be496b38f33928d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:49:06.651746 14081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a81579c8a2c466682938585880817f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:06.651945 13696 tablet_service.cc:1467] Processing CreateTablet for tablet 5a81579c8a2c466682938585880817f7 (DEFAULT_TABLE table=test-table [id=7dbf64438cf549a6be496b38f33928d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:49:06.651935 13890 tablet_service.cc:1467] Processing CreateTablet for tablet 5a81579c8a2c466682938585880817f7 (DEFAULT_TABLE table=test-table [id=7dbf64438cf549a6be496b38f33928d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:49:06.652324 13890 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a81579c8a2c466682938585880817f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:06.652346 13696 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a81579c8a2c466682938585880817f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:06.655766 14208 tablet_bootstrap.cc:492] T 5a81579c8a2c466682938585880817f7 P 72749adaae664ad7a1908ffca648d8b8: Bootstrap starting.
I20260806 22:49:06.656168 14212 tablet_bootstrap.cc:492] T 5a81579c8a2c466682938585880817f7 P cbab717dc326471ead3e4359e63c7c63: Bootstrap starting.
I20260806 22:49:06.656452 14211 tablet_bootstrap.cc:492] T 5a81579c8a2c466682938585880817f7 P 5864b7cd1ef347cb8fb7c63639e5f808: Bootstrap starting.
I20260806 22:49:06.656915 14212 tablet_bootstrap.cc:654] T 5a81579c8a2c466682938585880817f7 P cbab717dc326471ead3e4359e63c7c63: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:06.656968 14208 tablet_bootstrap.cc:654] T 5a81579c8a2c466682938585880817f7 P 72749adaae664ad7a1908ffca648d8b8: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:06.657423 14211 tablet_bootstrap.cc:654] T 5a81579c8a2c466682938585880817f7 P 5864b7cd1ef347cb8fb7c63639e5f808: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:06.657644 14212 log.cc:824] T 5a81579c8a2c466682938585880817f7 P cbab717dc326471ead3e4359e63c7c63: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:06.657784 14208 log.cc:824] T 5a81579c8a2c466682938585880817f7 P 72749adaae664ad7a1908ffca648d8b8: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:06.658242 14211 log.cc:824] T 5a81579c8a2c466682938585880817f7 P 5864b7cd1ef347cb8fb7c63639e5f808: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:06.658645 14212 tablet_bootstrap.cc:492] T 5a81579c8a2c466682938585880817f7 P cbab717dc326471ead3e4359e63c7c63: No bootstrap required, opened a new log
I20260806 22:49:06.658720 14208 tablet_bootstrap.cc:492] T 5a81579c8a2c466682938585880817f7 P 72749adaae664ad7a1908ffca648d8b8: No bootstrap required, opened a new log
I20260806 22:49:06.658753 14212 ts_tablet_manager.cc:1397] T 5a81579c8a2c466682938585880817f7 P cbab717dc326471ead3e4359e63c7c63: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:49:06.658797 14208 ts_tablet_manager.cc:1397] T 5a81579c8a2c466682938585880817f7 P 72749adaae664ad7a1908ffca648d8b8: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:49:06.659271 14211 tablet_bootstrap.cc:492] T 5a81579c8a2c466682938585880817f7 P 5864b7cd1ef347cb8fb7c63639e5f808: No bootstrap required, opened a new log
I20260806 22:49:06.659350 14211 ts_tablet_manager.cc:1397] T 5a81579c8a2c466682938585880817f7 P 5864b7cd1ef347cb8fb7c63639e5f808: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 22:49:06.660578 14208 raft_consensus.cc:348] T 5a81579c8a2c466682938585880817f7 P 72749adaae664ad7a1908ffca648d8b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5864b7cd1ef347cb8fb7c63639e5f808" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 33443 } } peers { permanent_uuid: "72749adaae664ad7a1908ffca648d8b8" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 32859 } } peers { permanent_uuid: "cbab717dc326471ead3e4359e63c7c63" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45865 } }
I20260806 22:49:06.660776 14208 raft_consensus.cc:374] T 5a81579c8a2c466682938585880817f7 P 72749adaae664ad7a1908ffca648d8b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:06.660826 14208 raft_consensus.cc:729] T 5a81579c8a2c466682938585880817f7 P 72749adaae664ad7a1908ffca648d8b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72749adaae664ad7a1908ffca648d8b8, State: Initialized, Role: FOLLOWER
I20260806 22:49:06.660969 14208 consensus_queue.cc:260] T 5a81579c8a2c466682938585880817f7 P 72749adaae664ad7a1908ffca648d8b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5864b7cd1ef347cb8fb7c63639e5f808" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 33443 } } peers { permanent_uuid: "72749adaae664ad7a1908ffca648d8b8" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 32859 } } peers { permanent_uuid: "cbab717dc326471ead3e4359e63c7c63" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45865 } }
I20260806 22:49:06.661329 14208 ts_tablet_manager.cc:1428] T 5a81579c8a2c466682938585880817f7 P 72749adaae664ad7a1908ffca648d8b8: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 22:49:06.661352 13784 heartbeater.cc:503] Master 127.18.45.254:42109 was elected leader, sending a full tablet report...
I20260806 22:49:06.661382 14212 raft_consensus.cc:348] T 5a81579c8a2c466682938585880817f7 P cbab717dc326471ead3e4359e63c7c63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5864b7cd1ef347cb8fb7c63639e5f808" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 33443 } } peers { permanent_uuid: "72749adaae664ad7a1908ffca648d8b8" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 32859 } } peers { permanent_uuid: "cbab717dc326471ead3e4359e63c7c63" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45865 } }
I20260806 22:49:06.661577 14212 raft_consensus.cc:374] T 5a81579c8a2c466682938585880817f7 P cbab717dc326471ead3e4359e63c7c63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:06.661618 14212 raft_consensus.cc:729] T 5a81579c8a2c466682938585880817f7 P cbab717dc326471ead3e4359e63c7c63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbab717dc326471ead3e4359e63c7c63, State: Initialized, Role: FOLLOWER
I20260806 22:49:06.661856 14212 consensus_queue.cc:260] T 5a81579c8a2c466682938585880817f7 P cbab717dc326471ead3e4359e63c7c63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5864b7cd1ef347cb8fb7c63639e5f808" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 33443 } } peers { permanent_uuid: "72749adaae664ad7a1908ffca648d8b8" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 32859 } } peers { permanent_uuid: "cbab717dc326471ead3e4359e63c7c63" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45865 } }
I20260806 22:49:06.662215 14212 ts_tablet_manager.cc:1428] T 5a81579c8a2c466682938585880817f7 P cbab717dc326471ead3e4359e63c7c63: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 22:49:06.662231 13977 heartbeater.cc:503] Master 127.18.45.254:42109 was elected leader, sending a full tablet report...
I20260806 22:49:06.662343 14211 raft_consensus.cc:348] T 5a81579c8a2c466682938585880817f7 P 5864b7cd1ef347cb8fb7c63639e5f808 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5864b7cd1ef347cb8fb7c63639e5f808" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 33443 } } peers { permanent_uuid: "72749adaae664ad7a1908ffca648d8b8" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 32859 } } peers { permanent_uuid: "cbab717dc326471ead3e4359e63c7c63" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45865 } }
I20260806 22:49:06.662540 14211 raft_consensus.cc:374] T 5a81579c8a2c466682938585880817f7 P 5864b7cd1ef347cb8fb7c63639e5f808 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:06.662599 14211 raft_consensus.cc:729] T 5a81579c8a2c466682938585880817f7 P 5864b7cd1ef347cb8fb7c63639e5f808 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5864b7cd1ef347cb8fb7c63639e5f808, State: Initialized, Role: FOLLOWER
I20260806 22:49:06.662737 14211 consensus_queue.cc:260] T 5a81579c8a2c466682938585880817f7 P 5864b7cd1ef347cb8fb7c63639e5f808 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5864b7cd1ef347cb8fb7c63639e5f808" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 33443 } } peers { permanent_uuid: "72749adaae664ad7a1908ffca648d8b8" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 32859 } } peers { permanent_uuid: "cbab717dc326471ead3e4359e63c7c63" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45865 } }
I20260806 22:49:06.663138 14211 ts_tablet_manager.cc:1428] T 5a81579c8a2c466682938585880817f7 P 5864b7cd1ef347cb8fb7c63639e5f808: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260806 22:49:06.663208 14176 heartbeater.cc:503] Master 127.18.45.254:42109 was elected leader, sending a full tablet report...
W20260806 22:49:06.667719 13980 tablet.cc:2369] T 5a81579c8a2c466682938585880817f7 P cbab717dc326471ead3e4359e63c7c63: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:49:06.693037 14217 raft_consensus.cc:482] T 5a81579c8a2c466682938585880817f7 P cbab717dc326471ead3e4359e63c7c63 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:49:06.693225 14217 raft_consensus.cc:504] T 5a81579c8a2c466682938585880817f7 P cbab717dc326471ead3e4359e63c7c63 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5864b7cd1ef347cb8fb7c63639e5f808" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 33443 } } peers { permanent_uuid: "72749adaae664ad7a1908ffca648d8b8" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 32859 } } peers { permanent_uuid: "cbab717dc326471ead3e4359e63c7c63" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45865 } }
I20260806 22:49:06.693724 14217 leader_election.cc:288] T 5a81579c8a2c466682938585880817f7 P cbab717dc326471ead3e4359e63c7c63 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5864b7cd1ef347cb8fb7c63639e5f808 (127.18.45.195:33443), 72749adaae664ad7a1908ffca648d8b8 (127.18.45.193:32859)
I20260806 22:49:06.698455 14204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:06.694155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:46357 (local address 127.18.45.193:32859)
0806 22:49:06.694239 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:06.694241 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:06.694252 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:49:06.694296 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:06.694297 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:06.694308 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:06.694363 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:49:06.694627 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:06.695512 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:06.697212 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:06.697916 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:06.698008 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:06.698022 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:06.698288 (+ 266us) server_negotiation.cc:299] Negotiation successful
0806 22:49:06.698336 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 22:49:06.698436 13981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:06.693996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.193:32859 (local address 127.18.45.194:46357)
0806 22:49:06.694099 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:06.694116 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:06.694179 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:06.694391 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:06.694392 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:06.694394 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:49:06.694505 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:06.694508 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:06.695649 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:06.695650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:06.697077 (+ 1427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:06.697089 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:06.698025 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:06.698043 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:06.698145 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:06.698150 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:49:06.698247 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 22:49:06.698309 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70}
I20260806 22:49:06.698596 14199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:06.694104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:39363 (local address 127.18.45.195:33443)
0806 22:49:06.694201 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:06.694203 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:06.694604 (+ 401us) server_negotiation.cc:407] Connection header received
0806 22:49:06.694762 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:06.694764 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:06.694784 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:06.694867 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:49:06.695244 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:06.695904 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:06.697441 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:06.698136 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:06.698199 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:06.698208 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:06.698441 (+ 233us) server_negotiation.cc:299] Negotiation successful
0806 22:49:06.698505 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 22:49:06.698621 14221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:06.694010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.195:33443 (local address 127.18.45.194:39363)
0806 22:49:06.694448 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:06.694460 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:06.694601 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:06.694985 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:06.694986 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:06.694989 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:49:06.695143 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:06.695145 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:06.695999 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:06.696000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:06.697282 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:06.697285 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:06.698242 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:06.698243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:06.698347 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:06.698351 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:49:06.698408 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 22:49:06.698445 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"mutex_wait_us":181,"thread_start_us":177,"threads_started":1}
I20260806 22:49:06.699021 13723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a81579c8a2c466682938585880817f7" candidate_uuid: "cbab717dc326471ead3e4359e63c7c63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72749adaae664ad7a1908ffca648d8b8" is_pre_election: true
I20260806 22:49:06.699213 13723 raft_consensus.cc:2454] T 5a81579c8a2c466682938585880817f7 P 72749adaae664ad7a1908ffca648d8b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cbab717dc326471ead3e4359e63c7c63 in term 0.
I20260806 22:49:06.699515 13817 leader_election.cc:302] T 5a81579c8a2c466682938585880817f7 P cbab717dc326471ead3e4359e63c7c63 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72749adaae664ad7a1908ffca648d8b8, cbab717dc326471ead3e4359e63c7c63; no voters:
I20260806 22:49:06.699466 14117 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a81579c8a2c466682938585880817f7" candidate_uuid: "cbab717dc326471ead3e4359e63c7c63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5864b7cd1ef347cb8fb7c63639e5f808" is_pre_election: true
I20260806 22:49:06.699693 14117 raft_consensus.cc:2454] T 5a81579c8a2c466682938585880817f7 P 5864b7cd1ef347cb8fb7c63639e5f808 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cbab717dc326471ead3e4359e63c7c63 in term 0.
I20260806 22:49:06.699769 14217 raft_consensus.cc:2781] T 5a81579c8a2c466682938585880817f7 P cbab717dc326471ead3e4359e63c7c63 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:49:06.699903 14217 raft_consensus.cc:482] T 5a81579c8a2c466682938585880817f7 P cbab717dc326471ead3e4359e63c7c63 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:49:06.699927 14217 raft_consensus.cc:3037] T 5a81579c8a2c466682938585880817f7 P cbab717dc326471ead3e4359e63c7c63 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:06.700914 14217 raft_consensus.cc:504] T 5a81579c8a2c466682938585880817f7 P cbab717dc326471ead3e4359e63c7c63 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5864b7cd1ef347cb8fb7c63639e5f808" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 33443 } } peers { permanent_uuid: "72749adaae664ad7a1908ffca648d8b8" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 32859 } } peers { permanent_uuid: "cbab717dc326471ead3e4359e63c7c63" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45865 } }
I20260806 22:49:06.701102 14217 leader_election.cc:288] T 5a81579c8a2c466682938585880817f7 P cbab717dc326471ead3e4359e63c7c63 [CANDIDATE]: Term 1 election: Requested vote from peers 5864b7cd1ef347cb8fb7c63639e5f808 (127.18.45.195:33443), 72749adaae664ad7a1908ffca648d8b8 (127.18.45.193:32859)
I20260806 22:49:06.701300 14117 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a81579c8a2c466682938585880817f7" candidate_uuid: "cbab717dc326471ead3e4359e63c7c63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5864b7cd1ef347cb8fb7c63639e5f808"
I20260806 22:49:06.701318 13723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a81579c8a2c466682938585880817f7" candidate_uuid: "cbab717dc326471ead3e4359e63c7c63" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72749adaae664ad7a1908ffca648d8b8"
I20260806 22:49:06.701393 14117 raft_consensus.cc:3037] T 5a81579c8a2c466682938585880817f7 P 5864b7cd1ef347cb8fb7c63639e5f808 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:06.701453 13723 raft_consensus.cc:3037] T 5a81579c8a2c466682938585880817f7 P 72749adaae664ad7a1908ffca648d8b8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:06.702405 14117 raft_consensus.cc:2454] T 5a81579c8a2c466682938585880817f7 P 5864b7cd1ef347cb8fb7c63639e5f808 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cbab717dc326471ead3e4359e63c7c63 in term 1.
I20260806 22:49:06.702497 13723 raft_consensus.cc:2454] T 5a81579c8a2c466682938585880817f7 P 72749adaae664ad7a1908ffca648d8b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cbab717dc326471ead3e4359e63c7c63 in term 1.
I20260806 22:49:06.702680 13816 leader_election.cc:302] T 5a81579c8a2c466682938585880817f7 P cbab717dc326471ead3e4359e63c7c63 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5864b7cd1ef347cb8fb7c63639e5f808, cbab717dc326471ead3e4359e63c7c63; no voters:
I20260806 22:49:06.702811 14217 raft_consensus.cc:2781] T 5a81579c8a2c466682938585880817f7 P cbab717dc326471ead3e4359e63c7c63 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:49:06.703100 14217 raft_consensus.cc:686] T 5a81579c8a2c466682938585880817f7 P cbab717dc326471ead3e4359e63c7c63 [term 1 LEADER]: Becoming Leader. State: Replica: cbab717dc326471ead3e4359e63c7c63, State: Running, Role: LEADER
I20260806 22:49:06.703241 14217 consensus_queue.cc:237] T 5a81579c8a2c466682938585880817f7 P cbab717dc326471ead3e4359e63c7c63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5864b7cd1ef347cb8fb7c63639e5f808" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 33443 } } peers { permanent_uuid: "72749adaae664ad7a1908ffca648d8b8" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 32859 } } peers { permanent_uuid: "cbab717dc326471ead3e4359e63c7c63" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45865 } }
I20260806 22:49:06.704480 13527 catalog_manager.cc:5495] T 5a81579c8a2c466682938585880817f7 P cbab717dc326471ead3e4359e63c7c63 reported cstate change: term changed from 0 to 1, leader changed from <none> to cbab717dc326471ead3e4359e63c7c63 (127.18.45.194). New cstate: current_term: 1 leader_uuid: "cbab717dc326471ead3e4359e63c7c63" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5864b7cd1ef347cb8fb7c63639e5f808" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 33443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72749adaae664ad7a1908ffca648d8b8" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 32859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cbab717dc326471ead3e4359e63c7c63" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 45865 } health_report { overall_health: HEALTHY } } }
I20260806 22:49:06.715415 14202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:06.712283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57126 (local address 127.18.45.194:45865)
0806 22:49:06.712353 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:06.712355 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:06.712365 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:49:06.712414 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:06.712415 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:06.712443 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:06.712530 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:49:06.712712 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:06.713275 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:06.714131 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:06.714356 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:06.714485 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:06.714796 (+ 311us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:49:06.714960 (+ 164us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:49:06.715196 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:06.715220 (+ 24us) server_negotiation.cc:299] Negotiation successful
0806 22:49:06.715252 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
W20260806 22:49:06.716223 13785 tablet.cc:2369] T 5a81579c8a2c466682938585880817f7 P 72749adaae664ad7a1908ffca648d8b8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:49:06.717739 13723 raft_consensus.cc:1261] T 5a81579c8a2c466682938585880817f7 P 72749adaae664ad7a1908ffca648d8b8 [term 1 FOLLOWER]: Refusing update from remote peer cbab717dc326471ead3e4359e63c7c63: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:49:06.717751 14117 raft_consensus.cc:1261] T 5a81579c8a2c466682938585880817f7 P 5864b7cd1ef347cb8fb7c63639e5f808 [term 1 FOLLOWER]: Refusing update from remote peer cbab717dc326471ead3e4359e63c7c63: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:49:06.718192 14222 consensus_queue.cc:1035] T 5a81579c8a2c466682938585880817f7 P cbab717dc326471ead3e4359e63c7c63 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72749adaae664ad7a1908ffca648d8b8" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 32859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:49:06.718379 14217 consensus_queue.cc:1035] T 5a81579c8a2c466682938585880817f7 P cbab717dc326471ead3e4359e63c7c63 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5864b7cd1ef347cb8fb7c63639e5f808" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 33443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:49:06.721321 14227 mvcc.cc:205] Tried to move back new op lower bound from 7315687615352537088 to 7315687615297318912. Current Snapshot: MvccSnapshot[applied={T|T < 7315687615352537088}]
I20260806 22:49:06.721768 14229 mvcc.cc:205] Tried to move back new op lower bound from 7315687615352537088 to 7315687615297318912. Current Snapshot: MvccSnapshot[applied={T|T < 7315687615352537088}]
I20260806 22:49:06.721767 14228 mvcc.cc:205] Tried to move back new op lower bound from 7315687615352537088 to 7315687615297318912. Current Snapshot: MvccSnapshot[applied={T|T < 7315687615352537088}]
I20260806 22:49:06.725029 13527 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58290:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:49:06.725167 13527 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58290:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:49:06.726197 13527 catalog_manager.cc:5782] T 00000000000000000000000000000000 P eab799275ad245db88c37299e74cc761: Sending DeleteTablet for 3 replicas of tablet 5a81579c8a2c466682938585880817f7
I20260806 22:49:06.726661 13890 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a81579c8a2c466682938585880817f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:49:06 UTC) from {username='kudu', principal='kudu/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:57110
I20260806 22:49:06.726663 14081 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a81579c8a2c466682938585880817f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:49:06 UTC) from {username='kudu', principal='kudu/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:33778
I20260806 22:49:06.726737 13696 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a81579c8a2c466682938585880817f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:49:06 UTC) from {username='kudu', principal='kudu/127.18.45.254@KRBTEST.COM'} at 127.0.0.1:54806
I20260806 22:49:06.727036 14241 tablet_replica.cc:330] T 5a81579c8a2c466682938585880817f7 P 72749adaae664ad7a1908ffca648d8b8: stopping tablet replica
I20260806 22:49:06.727047 14242 tablet_replica.cc:330] T 5a81579c8a2c466682938585880817f7 P cbab717dc326471ead3e4359e63c7c63: stopping tablet replica
I20260806 22:49:06.727164 14242 raft_consensus.cc:2229] T 5a81579c8a2c466682938585880817f7 P cbab717dc326471ead3e4359e63c7c63 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:49:06.727170 14241 raft_consensus.cc:2229] T 5a81579c8a2c466682938585880817f7 P 72749adaae664ad7a1908ffca648d8b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:49:06.727294 14241 raft_consensus.cc:2258] T 5a81579c8a2c466682938585880817f7 P 72749adaae664ad7a1908ffca648d8b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:49:06.727432 14240 tablet_replica.cc:330] T 5a81579c8a2c466682938585880817f7 P 5864b7cd1ef347cb8fb7c63639e5f808: stopping tablet replica
I20260806 22:49:06.727558 14240 raft_consensus.cc:2229] T 5a81579c8a2c466682938585880817f7 P 5864b7cd1ef347cb8fb7c63639e5f808 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:49:06.727676 14240 raft_consensus.cc:2258] T 5a81579c8a2c466682938585880817f7 P 5864b7cd1ef347cb8fb7c63639e5f808 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:49:06.727706 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 13609
I20260806 22:49:06.729043 14240 ts_tablet_manager.cc:1905] T 5a81579c8a2c466682938585880817f7 P 5864b7cd1ef347cb8fb7c63639e5f808: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:49:06.731768 14240 ts_tablet_manager.cc:1918] T 5a81579c8a2c466682938585880817f7 P 5864b7cd1ef347cb8fb7c63639e5f808: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:49:06.731868 14240 log.cc:1196] T 5a81579c8a2c466682938585880817f7 P 5864b7cd1ef347cb8fb7c63639e5f808: Deleting WAL directory at /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/minicluster-data/ts-2/wal/wals/5a81579c8a2c466682938585880817f7
I20260806 22:49:06.732137 14240 ts_tablet_manager.cc:1939] T 5a81579c8a2c466682938585880817f7 P 5864b7cd1ef347cb8fb7c63639e5f808: Deleting consensus metadata
I20260806 22:49:06.734226 13509 catalog_manager.cc:4841] TS 5864b7cd1ef347cb8fb7c63639e5f808 (127.18.45.195:33443): tablet 5a81579c8a2c466682938585880817f7 (table test-table [id=7dbf64438cf549a6be496b38f33928d5]) successfully deleted
I20260806 22:49:06.738888 14242 raft_consensus.cc:2258] T 5a81579c8a2c466682938585880817f7 P cbab717dc326471ead3e4359e63c7c63 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:49:06.739840 14242 ts_tablet_manager.cc:1905] T 5a81579c8a2c466682938585880817f7 P cbab717dc326471ead3e4359e63c7c63: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:49:06.742056 14242 ts_tablet_manager.cc:1918] T 5a81579c8a2c466682938585880817f7 P cbab717dc326471ead3e4359e63c7c63: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:49:06.742172 14242 log.cc:1196] T 5a81579c8a2c466682938585880817f7 P cbab717dc326471ead3e4359e63c7c63: Deleting WAL directory at /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786056275803658-18615-0/minicluster-data/ts-1/wal/wals/5a81579c8a2c466682938585880817f7
I20260806 22:49:06.742506 14242 ts_tablet_manager.cc:1939] T 5a81579c8a2c466682938585880817f7 P cbab717dc326471ead3e4359e63c7c63: Deleting consensus metadata
I20260806 22:49:06.743248 13511 catalog_manager.cc:4841] TS cbab717dc326471ead3e4359e63c7c63 (127.18.45.194:45865): tablet 5a81579c8a2c466682938585880817f7 (table test-table [id=7dbf64438cf549a6be496b38f33928d5]) successfully deleted
W20260806 22:49:06.745886 13510 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.45.193:32859 (error 108)
I20260806 22:49:06.746640 14195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:06.746457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.193:32859 (local address 127.0.0.1:54822)
0806 22:49:06.746534 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:06.746587 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.193:32859: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260806 22:49:06.746781 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 13789
W20260806 22:49:06.747051 13510 catalog_manager.cc:4584] TS 72749adaae664ad7a1908ffca648d8b8 (127.18.45.193:32859): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5a81579c8a2c466682938585880817f7: Network error: Client connection negotiation failed: client connection to 127.18.45.193:32859: connect: Connection refused (error 111)
I20260806 22:49:06.765216 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 13986
I20260806 22:49:06.777912 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 13481
2026-08-06T22:49:06Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2437 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-06T22:49:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:49:06Z Disabled control of system clock
I20260806 22:49:06.821028 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056275803658-18615-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056275803658-18615-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056275803658-18615-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:39681
--webserver_interface=127.18.45.254
--webserver_port=0
--builtin_ntp_servers=127.18.45.212:43617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.45.254:39681
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 22:49:06.924232 14247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:06.924540 14247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:06.924566 14247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:06.925927 14247 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:49:06.925982 14247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:06.925995 14247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:06.926008 14247 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:49:06.926021 14247 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:49:06.927479 14247 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:43617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056275803658-18615-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056275803658-18615-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.45.254:39681
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:39681
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.45.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14247
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056275803658-18615-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:49:06.927765 14247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:06.928103 14247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:06.934093 14257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:06.934093 14253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:06.934204 14254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:06.934260 14247 server_base.cc:1034] running on GCE node
I20260806 22:49:06.934814 14247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:06.935290 14247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:06.936478 14247 hybrid_clock.cc:648] HybridClock initialized: now 1786056546936454 us; error 44 us; skew 500 ppm
I20260806 22:49:06.938118 14247 webserver.cc:460] Webserver started at http://127.18.45.254:38165/ using document root <none> and password file <none>
I20260806 22:49:06.938416 14247 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:06.938491 14247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:06.938673 14247 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:49:06.940127 14247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056275803658-18615-0/minicluster-data/master-0/data/instance:
uuid: "5d9eb43716d34e02b5769d683bdca833"
format_stamp: "Formatted at 2026-08-06 22:49:06 on dist-test-slave-h3ft"
server_key: "71e46b38735c73f7057c89b69f6e6772"
server_key_iv: "9fc06ecbae73f224fb1ded1021151982"
server_key_version: "encryptionkey@0"
I20260806 22:49:06.940586 14247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056275803658-18615-0/minicluster-data/master-0/wal/instance:
uuid: "5d9eb43716d34e02b5769d683bdca833"
format_stamp: "Formatted at 2026-08-06 22:49:06 on dist-test-slave-h3ft"
server_key: "71e46b38735c73f7057c89b69f6e6772"
server_key_iv: "9fc06ecbae73f224fb1ded1021151982"
server_key_version: "encryptionkey@0"
I20260806 22:49:06.942220 14247 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:49:06.943434 14268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:06.943642 14247 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:49:06.943759 14247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056275803658-18615-0/minicluster-data/master-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056275803658-18615-0/minicluster-data/master-0/wal
uuid: "5d9eb43716d34e02b5769d683bdca833"
format_stamp: "Formatted at 2026-08-06 22:49:06 on dist-test-slave-h3ft"
server_key: "71e46b38735c73f7057c89b69f6e6772"
server_key_iv: "9fc06ecbae73f224fb1ded1021151982"
server_key_version: "encryptionkey@0"
I20260806 22:49:06.943831 14247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056275803658-18615-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056275803658-18615-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056275803658-18615-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:06.954933 14247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:06.955448 14247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:06.955637 14247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:06.961199 14247 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.254:39681
I20260806 22:49:06.961323 14338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.254:39681 every 8 connection(s)
I20260806 22:49:06.961768 14247 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
I20260806 22:49:06.963193 14339 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:06.965993 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 14247
I20260806 22:49:06.966153 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056275803658-18615-0/minicluster-data/master-0/wal/instance
I20260806 22:49:06.966429 18615 external_mini_cluster.cc:1428] Setting key 5bce4112597659dd2f56a39cb5444d58
I20260806 22:49:06.966804 14339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d9eb43716d34e02b5769d683bdca833: Bootstrap starting.
I20260806 22:49:06.967824 14339 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5d9eb43716d34e02b5769d683bdca833: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:06.968247 14339 log.cc:824] T 00000000000000000000000000000000 P 5d9eb43716d34e02b5769d683bdca833: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:06.969364 14339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d9eb43716d34e02b5769d683bdca833: No bootstrap required, opened a new log
I20260806 22:49:06.972033 14339 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d9eb43716d34e02b5769d683bdca833 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d9eb43716d34e02b5769d683bdca833" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39681 } }
I20260806 22:49:06.972275 14339 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5d9eb43716d34e02b5769d683bdca833 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:06.972311 14339 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d9eb43716d34e02b5769d683bdca833 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d9eb43716d34e02b5769d683bdca833, State: Initialized, Role: FOLLOWER
I20260806 22:49:06.972491 14339 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d9eb43716d34e02b5769d683bdca833 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d9eb43716d34e02b5769d683bdca833" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39681 } }
I20260806 22:49:06.972623 14339 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5d9eb43716d34e02b5769d683bdca833 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:49:06.972666 14339 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5d9eb43716d34e02b5769d683bdca833 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:49:06.972721 14339 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d9eb43716d34e02b5769d683bdca833 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:06.973603 14339 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5d9eb43716d34e02b5769d683bdca833 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d9eb43716d34e02b5769d683bdca833" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39681 } }
I20260806 22:49:06.973752 14339 leader_election.cc:302] T 00000000000000000000000000000000 P 5d9eb43716d34e02b5769d683bdca833 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5d9eb43716d34e02b5769d683bdca833; no voters:
I20260806 22:49:06.973815 14346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:06.967246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48932 (local address 127.18.45.254:39681)
0806 22:49:06.967536 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:06.967546 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:06.967577 (+ 31us) server_negotiation.cc:407] Connection header received
0806 22:49:06.968212 (+ 635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:06.968222 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:06.968466 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:06.968693 (+ 227us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:06.968966 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:06.970023 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:06.971266 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:06.971760 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:06.971882 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:06.972937 (+ 1055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:06.972942 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:06.972942 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:49:06.973159 (+ 217us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:06.973206 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:49:06.973219 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:06.973278 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:06.973393 (+ 115us) server_negotiation.cc:299] Negotiation successful
0806 22:49:06.973468 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":143,"threads_started":1}
I20260806 22:49:06.974017 14339 leader_election.cc:288] T 00000000000000000000000000000000 P 5d9eb43716d34e02b5769d683bdca833 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:49:06.974051 14348 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5d9eb43716d34e02b5769d683bdca833 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:49:06.974189 14348 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5d9eb43716d34e02b5769d683bdca833 [term 1 LEADER]: Becoming Leader. State: Replica: 5d9eb43716d34e02b5769d683bdca833, State: Running, Role: LEADER
I20260806 22:49:06.974288 14348 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5d9eb43716d34e02b5769d683bdca833 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d9eb43716d34e02b5769d683bdca833" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39681 } }
I20260806 22:49:06.974391 14339 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d9eb43716d34e02b5769d683bdca833 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:49:06.974684 14348 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d9eb43716d34e02b5769d683bdca833 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5d9eb43716d34e02b5769d683bdca833" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d9eb43716d34e02b5769d683bdca833" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39681 } } }
I20260806 22:49:06.974761 14348 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d9eb43716d34e02b5769d683bdca833 [sys.catalog]: This master's current role is: LEADER
I20260806 22:49:06.974916 14349 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d9eb43716d34e02b5769d683bdca833 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d9eb43716d34e02b5769d683bdca833. Latest consensus state: current_term: 1 leader_uuid: "5d9eb43716d34e02b5769d683bdca833" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d9eb43716d34e02b5769d683bdca833" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 39681 } } }
I20260806 22:49:06.974979 14349 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d9eb43716d34e02b5769d683bdca833 [sys.catalog]: This master's current role is: LEADER
I20260806 22:49:06.975878 14354 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:49:06.976473 14354 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:49:06.978227 14354 catalog_manager.cc:1347] Generated new cluster ID: 1d8370c488d646e992e0ca285e1a4d6e
I20260806 22:49:06.978289 14354 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:49:06.994588 14354 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:49:06.995344 14354 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:49:07.000394 14354 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5d9eb43716d34e02b5769d683bdca833: Generated new TSK 0
I20260806 22:49:07.000694 14354 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:49:07.009277 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056275803658-18615-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056275803658-18615-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056275803658-18615-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.193:0
--local_ip_for_outbound_sockets=127.18.45.193
--webserver_interface=127.18.45.193
--webserver_port=0
--tserver_master_addrs=127.18.45.254:39681
--builtin_ntp_servers=127.18.45.212:43617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 22:49:07.111742 14373 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:07.112023 14373 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:07.112051 14373 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:07.113420 14373 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:07.113487 14373 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:07.113517 14373 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.193
I20260806 22:49:07.115140 14373 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:43617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056275803658-18615-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056275803658-18615-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.45.193
--webserver_port=0
--tserver_master_addrs=127.18.45.254:39681
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14373
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.193
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056275803658-18615-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:49:07.115463 14373 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:07.115854 14373 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:07.122066 14386 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:07.122078 14382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:07.122092 14383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:07.122478 14373 server_base.cc:1034] running on GCE node
I20260806 22:49:07.122743 14373 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:07.123095 14373 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:07.124291 14373 hybrid_clock.cc:648] HybridClock initialized: now 1786056547124264 us; error 45 us; skew 500 ppm
I20260806 22:49:07.126022 14373 webserver.cc:460] Webserver started at http://127.18.45.193:39903/ using document root <none> and password file <none>
I20260806 22:49:07.126356 14373 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:07.126438 14373 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:07.126614 14373 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:49:07.128074 14373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056275803658-18615-0/minicluster-data/ts-0/data/instance:
uuid: "a0e72d89ce6943819b36930b465ba29d"
format_stamp: "Formatted at 2026-08-06 22:49:07 on dist-test-slave-h3ft"
server_key: "72ec335879386e2a4f4758a3ebba2406"
server_key_iv: "b23fe5d17216a1ade64f25a7d5ca4482"
server_key_version: "encryptionkey@0"
I20260806 22:49:07.128517 14373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056275803658-18615-0/minicluster-data/ts-0/wal/instance:
uuid: "a0e72d89ce6943819b36930b465ba29d"
format_stamp: "Formatted at 2026-08-06 22:49:07 on dist-test-slave-h3ft"
server_key: "72ec335879386e2a4f4758a3ebba2406"
server_key_iv: "b23fe5d17216a1ade64f25a7d5ca4482"
server_key_version: "encryptionkey@0"
I20260806 22:49:07.130185 14373 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260806 22:49:07.131170 14392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:07.131394 14373 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:49:07.131510 14373 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056275803658-18615-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056275803658-18615-0/minicluster-data/ts-0/wal
uuid: "a0e72d89ce6943819b36930b465ba29d"
format_stamp: "Formatted at 2026-08-06 22:49:07 on dist-test-slave-h3ft"
server_key: "72ec335879386e2a4f4758a3ebba2406"
server_key_iv: "b23fe5d17216a1ade64f25a7d5ca4482"
server_key_version: "encryptionkey@0"
I20260806 22:49:07.131601 14373 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056275803658-18615-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056275803658-18615-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056275803658-18615-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:07.145186 14373 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:07.145605 14373 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:07.145789 14373 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:07.146083 14373 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:49:07.146517 14373 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:49:07.146587 14373 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:07.146637 14373 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:49:07.146668 14373 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:07.154949 14373 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.193:37787
I20260806 22:49:07.154999 14544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.193:37787 every 8 connection(s)
I20260806 22:49:07.155462 14373 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056275803658-18615-0/minicluster-data/ts-0/data/info.pb
I20260806 22:49:07.157809 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 14373
I20260806 22:49:07.158020 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056275803658-18615-0/minicluster-data/ts-0/wal/instance
I20260806 22:49:07.158277 18615 external_mini_cluster.cc:1428] Setting key 58c6197253124400656d7289c1900e2c
I20260806 22:49:07.159616 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056275803658-18615-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056275803658-18615-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056275803658-18615-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.194:0
--local_ip_for_outbound_sockets=127.18.45.194
--webserver_interface=127.18.45.194
--webserver_port=0
--tserver_master_addrs=127.18.45.254:39681
--builtin_ntp_servers=127.18.45.212:43617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 22:49:07.162837 14346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:07.156400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.193:38723 (local address 127.18.45.254:39681)
0806 22:49:07.156490 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:07.156493 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:07.156801 (+ 308us) server_negotiation.cc:407] Connection header received
0806 22:49:07.157521 (+ 720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:07.157522 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:07.157536 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:07.157645 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:07.158683 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:07.159360 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:07.160998 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:07.161202 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:07.161274 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:07.162471 (+ 1197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:07.162473 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:07.162474 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:49:07.162482 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:07.162505 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:49:07.162510 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:07.162577 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:07.162675 (+ 98us) server_negotiation.cc:299] Negotiation successful
0806 22:49:07.162711 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 22:49:07.163070 14547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:07.156391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39681 (local address 127.18.45.193:38723)
0806 22:49:07.156651 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:07.156697 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:07.157374 (+ 677us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:07.157733 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:07.157738 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:07.157748 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:07.158529 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:07.158536 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:07.159457 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:07.159458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:07.160871 (+ 1413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:07.160875 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:07.161334 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:07.161335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:07.161557 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:07.162248 (+ 691us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:49:07.162249 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:07.162285 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:49:07.162314 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:07.162585 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:07.162591 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:07.162594 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:49:07.162654 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 22:49:07.162759 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":106,"threads_started":1}
I20260806 22:49:07.163648 14545 heartbeater.cc:347] Connected to a master server at 127.18.45.254:39681
I20260806 22:49:07.163776 14545 heartbeater.cc:464] Registering TS with master...
I20260806 22:49:07.163996 14545 heartbeater.cc:511] Master 127.18.45.254:39681 requested a full tablet report, sending...
I20260806 22:49:07.164680 14288 ts_manager.cc:194] Registered new tserver with Master: a0e72d89ce6943819b36930b465ba29d (127.18.45.193:37787)
I20260806 22:49:07.165401 14288 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.45.193:38723
W20260806 22:49:07.259714 14548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:07.259982 14548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:07.260007 14548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:07.261304 14548 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:07.261356 14548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:07.261384 14548 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.194
I20260806 22:49:07.262949 14548 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:43617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056275803658-18615-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056275803658-18615-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.45.194
--webserver_port=0
--tserver_master_addrs=127.18.45.254:39681
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14548
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.194
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056275803658-18615-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:49:07.263285 14548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:07.263652 14548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:07.270102 14556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:07.270087 14554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:07.270128 14560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:07.270313 14548 server_base.cc:1034] running on GCE node
I20260806 22:49:07.270583 14548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:07.270887 14548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:07.272080 14548 hybrid_clock.cc:648] HybridClock initialized: now 1786056547272069 us; error 33 us; skew 500 ppm
I20260806 22:49:07.273900 14548 webserver.cc:460] Webserver started at http://127.18.45.194:42691/ using document root <none> and password file <none>
I20260806 22:49:07.274240 14548 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:07.274298 14548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:07.274461 14548 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:49:07.276000 14548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056275803658-18615-0/minicluster-data/ts-1/data/instance:
uuid: "901eb508a51f4206bb28171ad7fcefde"
format_stamp: "Formatted at 2026-08-06 22:49:07 on dist-test-slave-h3ft"
server_key: "c2254570a6ff96e0e979eeb26b61a433"
server_key_iv: "0af441a5ea409899724811b9523cbd4c"
server_key_version: "encryptionkey@0"
I20260806 22:49:07.276450 14548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056275803658-18615-0/minicluster-data/ts-1/wal/instance:
uuid: "901eb508a51f4206bb28171ad7fcefde"
format_stamp: "Formatted at 2026-08-06 22:49:07 on dist-test-slave-h3ft"
server_key: "c2254570a6ff96e0e979eeb26b61a433"
server_key_iv: "0af441a5ea409899724811b9523cbd4c"
server_key_version: "encryptionkey@0"
I20260806 22:49:07.278201 14548 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260806 22:49:07.279299 14568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:07.279546 14548 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:49:07.279656 14548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056275803658-18615-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056275803658-18615-0/minicluster-data/ts-1/wal
uuid: "901eb508a51f4206bb28171ad7fcefde"
format_stamp: "Formatted at 2026-08-06 22:49:07 on dist-test-slave-h3ft"
server_key: "c2254570a6ff96e0e979eeb26b61a433"
server_key_iv: "0af441a5ea409899724811b9523cbd4c"
server_key_version: "encryptionkey@0"
I20260806 22:49:07.279745 14548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056275803658-18615-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056275803658-18615-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056275803658-18615-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:07.293617 14548 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:07.294059 14548 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:07.294236 14548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:07.294544 14548 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:49:07.294965 14548 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:49:07.295073 14548 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:07.295126 14548 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:49:07.295152 14548 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:07.302850 14548 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.194:44095
I20260806 22:49:07.302909 14725 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.194:44095 every 8 connection(s)
I20260806 22:49:07.303382 14548 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056275803658-18615-0/minicluster-data/ts-1/data/info.pb
I20260806 22:49:07.308387 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 14548
I20260806 22:49:07.308557 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056275803658-18615-0/minicluster-data/ts-1/wal/instance
I20260806 22:49:07.308825 18615 external_mini_cluster.cc:1428] Setting key e80f6f5a8cd5bccac353c498414b8e19
I20260806 22:49:07.309962 14346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:07.304361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.194:34039 (local address 127.18.45.254:39681)
0806 22:49:07.304456 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:07.304459 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:07.304727 (+ 268us) server_negotiation.cc:407] Connection header received
0806 22:49:07.305332 (+ 605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:07.305334 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:07.305366 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:07.305438 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:07.306331 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:07.306923 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:07.308015 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:07.308223 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:07.308285 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:07.309609 (+ 1324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:07.309611 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:07.309612 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:49:07.309618 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:07.309641 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:49:07.309646 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:07.309713 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:07.309856 (+ 143us) server_negotiation.cc:299] Negotiation successful
0806 22:49:07.309886 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 22:49:07.310141 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056275803658-18615-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056275803658-18615-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056275803658-18615-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.45.195:0
--local_ip_for_outbound_sockets=127.18.45.195
--webserver_interface=127.18.45.195
--webserver_port=0
--tserver_master_addrs=127.18.45.254:39681
--builtin_ntp_servers=127.18.45.212:43617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 22:49:07.310293 14730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:07.304327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39681 (local address 127.18.45.194:34039)
0806 22:49:07.304561 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:07.304611 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:07.305223 (+ 612us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:07.305534 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:07.305539 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:07.305549 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:07.306226 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:07.306232 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:07.307046 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:07.307047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:07.307921 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:07.307923 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:07.308323 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:07.308324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:07.308514 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:07.309428 (+ 914us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:49:07.309428 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:07.309466 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:49:07.309499 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:07.309741 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:07.309751 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:07.309755 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:49:07.309843 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 22:49:07.309990 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":118,"threads_started":1}
I20260806 22:49:07.310804 14726 heartbeater.cc:347] Connected to a master server at 127.18.45.254:39681
I20260806 22:49:07.311074 14726 heartbeater.cc:464] Registering TS with master...
I20260806 22:49:07.311331 14726 heartbeater.cc:511] Master 127.18.45.254:39681 requested a full tablet report, sending...
I20260806 22:49:07.311823 14288 ts_manager.cc:194] Registered new tserver with Master: 901eb508a51f4206bb28171ad7fcefde (127.18.45.194:44095)
I20260806 22:49:07.312382 14288 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.45.194:34039
W20260806 22:49:07.409093 14731 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:07.409353 14731 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:07.409379 14731 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:07.410658 14731 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:07.410702 14731 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:07.410729 14731 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.45.195
I20260806 22:49:07.412245 14731 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:43617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056275803658-18615-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056275803658-18615-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.45.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.45.195
--webserver_port=0
--tserver_master_addrs=127.18.45.254:39681
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14731
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.45.195
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056275803658-18615-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:49:07.412552 14731 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:07.412978 14731 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:07.418807 14739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:07.418862 14742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:07.418841 14740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:07.418994 14731 server_base.cc:1034] running on GCE node
I20260806 22:49:07.419298 14731 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:07.419586 14731 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:07.420743 14731 hybrid_clock.cc:648] HybridClock initialized: now 1786056547420728 us; error 30 us; skew 500 ppm
I20260806 22:49:07.422340 14731 webserver.cc:460] Webserver started at http://127.18.45.195:33189/ using document root <none> and password file <none>
I20260806 22:49:07.422668 14731 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:07.422748 14731 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:07.422920 14731 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:49:07.424237 14731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056275803658-18615-0/minicluster-data/ts-2/data/instance:
uuid: "6001fbb534ca49bf960cc6d681df7958"
format_stamp: "Formatted at 2026-08-06 22:49:07 on dist-test-slave-h3ft"
server_key: "f3a5c167fe49e0a57cf5758a63e25a6e"
server_key_iv: "a2d93e5f8f5bef7de543cb7c02764561"
server_key_version: "encryptionkey@0"
I20260806 22:49:07.424655 14731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056275803658-18615-0/minicluster-data/ts-2/wal/instance:
uuid: "6001fbb534ca49bf960cc6d681df7958"
format_stamp: "Formatted at 2026-08-06 22:49:07 on dist-test-slave-h3ft"
server_key: "f3a5c167fe49e0a57cf5758a63e25a6e"
server_key_iv: "a2d93e5f8f5bef7de543cb7c02764561"
server_key_version: "encryptionkey@0"
I20260806 22:49:07.426216 14731 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:49:07.427161 14750 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:07.427364 14731 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:49:07.427479 14731 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056275803658-18615-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056275803658-18615-0/minicluster-data/ts-2/wal
uuid: "6001fbb534ca49bf960cc6d681df7958"
format_stamp: "Formatted at 2026-08-06 22:49:07 on dist-test-slave-h3ft"
server_key: "f3a5c167fe49e0a57cf5758a63e25a6e"
server_key_iv: "a2d93e5f8f5bef7de543cb7c02764561"
server_key_version: "encryptionkey@0"
I20260806 22:49:07.427573 14731 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056275803658-18615-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056275803658-18615-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056275803658-18615-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:07.445308 14731 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:07.445740 14731 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:07.445917 14731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:07.446254 14731 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 22:49:07.446702 14731 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 22:49:07.446763 14731 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:07.446825 14731 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 22:49:07.446847 14731 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:07.454459 14731 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.195:40663
I20260806 22:49:07.454527 14905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.195:40663 every 8 connection(s)
I20260806 22:49:07.454963 14731 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056275803658-18615-0/minicluster-data/ts-2/data/info.pb
I20260806 22:49:07.458247 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 14731
I20260806 22:49:07.458411 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056275803658-18615-0/minicluster-data/ts-2/wal/instance
I20260806 22:49:07.458633 18615 external_mini_cluster.cc:1428] Setting key d98feb4dd463ca8f56df5fa049c87044
I20260806 22:49:07.461660 14346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:07.456009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.195:47055 (local address 127.18.45.254:39681)
0806 22:49:07.456135 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:07.456138 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:07.456395 (+ 257us) server_negotiation.cc:407] Connection header received
0806 22:49:07.457124 (+ 729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:07.457126 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:07.457141 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:07.457200 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:07.458054 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:07.458741 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:07.459904 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:07.460127 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:07.460218 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:07.461294 (+ 1076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:07.461296 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:07.461297 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:49:07.461302 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:07.461331 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:49:07.461335 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:07.461406 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:07.461519 (+ 113us) server_negotiation.cc:299] Negotiation successful
0806 22:49:07.461559 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 22:49:07.461969 14908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:07.455903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:39681 (local address 127.18.45.195:47055)
0806 22:49:07.456246 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:07.456289 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:07.457000 (+ 711us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:07.457296 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:07.457300 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:07.457311 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:07.457930 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:07.457935 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:07.458831 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:07.458833 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:07.459786 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:07.459788 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:07.460230 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:07.460232 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:07.460401 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:07.461116 (+ 715us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 22:49:07.461117 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:07.461154 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 22:49:07.461183 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:07.461406 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:07.461415 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:07.461418 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 22:49:07.461520 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 22:49:07.461655 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":183,"threads_started":1}
I20260806 22:49:07.462726 14906 heartbeater.cc:347] Connected to a master server at 127.18.45.254:39681
I20260806 22:49:07.462857 14906 heartbeater.cc:464] Registering TS with master...
I20260806 22:49:07.463189 14906 heartbeater.cc:511] Master 127.18.45.254:39681 requested a full tablet report, sending...
I20260806 22:49:07.463752 14288 ts_manager.cc:194] Registered new tserver with Master: 6001fbb534ca49bf960cc6d681df7958 (127.18.45.195:40663)
I20260806 22:49:07.464177 14288 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.45.195:47055
I20260806 22:49:07.470882 18615 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 22:49:07.476529 14346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:07.472801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48948 (local address 127.18.45.254:39681)
0806 22:49:07.472883 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:07.472885 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:07.473069 (+ 184us) server_negotiation.cc:407] Connection header received
0806 22:49:07.473133 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:07.473134 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:07.473148 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:07.473207 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:07.473393 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:07.473934 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:07.474905 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:07.475221 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:07.475262 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:07.476198 (+ 936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:07.476200 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:07.476200 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:49:07.476206 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:07.476223 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:49:07.476227 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:07.476317 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:07.476383 (+ 66us) server_negotiation.cc:299] Negotiation successful
0806 22:49:07.476414 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 22:49:07.478459 14288 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48948:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 22:49:07.480538 14288 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 22:49:07.488067 14919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:07.483458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41406 (local address 127.18.45.193:37787)
0806 22:49:07.483956 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:07.483963 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:07.483976 (+ 13us) server_negotiation.cc:407] Connection header received
0806 22:49:07.484025 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:07.484027 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:07.484046 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:07.484169 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:49:07.484448 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:07.485328 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:07.486954 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:07.487574 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:07.487678 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:07.487797 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:07.487873 (+ 76us) server_negotiation.cc:299] Negotiation successful
0806 22:49:07.487917 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":120,"threads_started":1}
I20260806 22:49:07.488066 14915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:07.483239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.195:40663 (local address 127.0.0.1:47878)
0806 22:49:07.483398 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:07.483448 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:07.483525 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:07.484564 (+ 1039us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:07.484565 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:07.484568 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:49:07.484618 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:07.484619 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:07.485874 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:07.485875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:07.486744 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:07.486747 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:07.487786 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:07.487788 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:07.487857 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:07.487863 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 22:49:07.487897 (+ 34us) client_negotiation.cc:240] Negotiation successful
0806 22:49:07.487946 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":80,"threads_started":1}
I20260806 22:49:07.488058 14918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:07.483240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.193:37787 (local address 127.0.0.1:41406)
0806 22:49:07.483703 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:07.483716 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:07.483777 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:07.484212 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:07.484215 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:07.484218 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:49:07.484356 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:07.484359 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:07.485416 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:07.485417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:07.486849 (+ 1432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:07.486852 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:07.487681 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:07.487682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:07.487762 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:07.487776 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 22:49:07.487856 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 22:49:07.487920 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"mutex_wait_us":48,"thread_start_us":60,"threads_started":1}
I20260806 22:49:07.488719 14920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:07.483572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47878 (local address 127.18.45.195:40663)
0806 22:49:07.484384 (+ 812us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:07.484389 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:07.484400 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:49:07.484449 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:07.484450 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:07.484468 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:07.484559 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:49:07.484684 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:07.485796 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:07.486843 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:07.487674 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:07.488113 (+ 439us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:07.488540 (+ 427us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:07.488568 (+ 28us) server_negotiation.cc:299] Negotiation successful
0806 22:49:07.488619 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":745,"thread_start_us":125,"threads_started":1}
I20260806 22:49:07.489054 14458 tablet_service.cc:1467] Processing CreateTablet for tablet 1174265abf4a4444bcbef2c7a66a2be7 (DEFAULT_TABLE table=test-table [id=ab65a706a44d4b008026bcb542f4460b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:49:07.489223 14926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:07.483752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34234 (local address 127.18.45.194:44095)
0806 22:49:07.484016 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:07.484021 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:07.484044 (+ 23us) server_negotiation.cc:407] Connection header received
0806 22:49:07.484155 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:07.484158 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:07.484176 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:07.484264 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:49:07.484476 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:07.485725 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:07.487034 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:07.487808 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:07.487888 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:07.487987 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:07.489075 (+ 1088us) server_negotiation.cc:299] Negotiation successful
0806 22:49:07.489130 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":111,"threads_started":1}
I20260806 22:49:07.488782 14917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:07.483521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.194:44095 (local address 127.0.0.1:34234)
0806 22:49:07.483765 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:07.483776 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:07.484025 (+ 249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:07.484278 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:07.484279 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:07.484282 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:49:07.484366 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:07.484369 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:07.485821 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:07.485823 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:07.486921 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:07.486923 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:07.488025 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:07.488027 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:07.488092 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:07.488097 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 22:49:07.488145 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 22:49:07.488698 (+ 553us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"mutex_wait_us":10,"thread_start_us":58,"threads_started":1}
I20260806 22:49:07.489545 14458 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1174265abf4a4444bcbef2c7a66a2be7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:07.489451 14818 tablet_service.cc:1467] Processing CreateTablet for tablet 1174265abf4a4444bcbef2c7a66a2be7 (DEFAULT_TABLE table=test-table [id=ab65a706a44d4b008026bcb542f4460b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:49:07.489871 14818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1174265abf4a4444bcbef2c7a66a2be7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:07.491748 14636 tablet_service.cc:1467] Processing CreateTablet for tablet 1174265abf4a4444bcbef2c7a66a2be7 (DEFAULT_TABLE table=test-table [id=ab65a706a44d4b008026bcb542f4460b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 22:49:07.492161 14636 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1174265abf4a4444bcbef2c7a66a2be7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:07.493182 14927 tablet_bootstrap.cc:492] T 1174265abf4a4444bcbef2c7a66a2be7 P a0e72d89ce6943819b36930b465ba29d: Bootstrap starting.
I20260806 22:49:07.493619 14928 tablet_bootstrap.cc:492] T 1174265abf4a4444bcbef2c7a66a2be7 P 6001fbb534ca49bf960cc6d681df7958: Bootstrap starting.
I20260806 22:49:07.494202 14927 tablet_bootstrap.cc:654] T 1174265abf4a4444bcbef2c7a66a2be7 P a0e72d89ce6943819b36930b465ba29d: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:07.494457 14928 tablet_bootstrap.cc:654] T 1174265abf4a4444bcbef2c7a66a2be7 P 6001fbb534ca49bf960cc6d681df7958: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:07.494647 14927 log.cc:824] T 1174265abf4a4444bcbef2c7a66a2be7 P a0e72d89ce6943819b36930b465ba29d: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:07.494841 14928 log.cc:824] T 1174265abf4a4444bcbef2c7a66a2be7 P 6001fbb534ca49bf960cc6d681df7958: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:07.495754 14928 tablet_bootstrap.cc:492] T 1174265abf4a4444bcbef2c7a66a2be7 P 6001fbb534ca49bf960cc6d681df7958: No bootstrap required, opened a new log
I20260806 22:49:07.495818 14928 ts_tablet_manager.cc:1397] T 1174265abf4a4444bcbef2c7a66a2be7 P 6001fbb534ca49bf960cc6d681df7958: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260806 22:49:07.496186 14931 tablet_bootstrap.cc:492] T 1174265abf4a4444bcbef2c7a66a2be7 P 901eb508a51f4206bb28171ad7fcefde: Bootstrap starting.
I20260806 22:49:07.497066 14931 tablet_bootstrap.cc:654] T 1174265abf4a4444bcbef2c7a66a2be7 P 901eb508a51f4206bb28171ad7fcefde: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:07.497246 14927 tablet_bootstrap.cc:492] T 1174265abf4a4444bcbef2c7a66a2be7 P a0e72d89ce6943819b36930b465ba29d: No bootstrap required, opened a new log
I20260806 22:49:07.497340 14927 ts_tablet_manager.cc:1397] T 1174265abf4a4444bcbef2c7a66a2be7 P a0e72d89ce6943819b36930b465ba29d: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 22:49:07.497515 14931 log.cc:824] T 1174265abf4a4444bcbef2c7a66a2be7 P 901eb508a51f4206bb28171ad7fcefde: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:07.498368 14931 tablet_bootstrap.cc:492] T 1174265abf4a4444bcbef2c7a66a2be7 P 901eb508a51f4206bb28171ad7fcefde: No bootstrap required, opened a new log
I20260806 22:49:07.498484 14931 ts_tablet_manager.cc:1397] T 1174265abf4a4444bcbef2c7a66a2be7 P 901eb508a51f4206bb28171ad7fcefde: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 22:49:07.498944 14928 raft_consensus.cc:348] T 1174265abf4a4444bcbef2c7a66a2be7 P 6001fbb534ca49bf960cc6d681df7958 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6001fbb534ca49bf960cc6d681df7958" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 40663 } } peers { permanent_uuid: "a0e72d89ce6943819b36930b465ba29d" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 37787 } } peers { permanent_uuid: "901eb508a51f4206bb28171ad7fcefde" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 44095 } }
I20260806 22:49:07.499264 14928 raft_consensus.cc:374] T 1174265abf4a4444bcbef2c7a66a2be7 P 6001fbb534ca49bf960cc6d681df7958 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:07.499310 14928 raft_consensus.cc:729] T 1174265abf4a4444bcbef2c7a66a2be7 P 6001fbb534ca49bf960cc6d681df7958 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6001fbb534ca49bf960cc6d681df7958, State: Initialized, Role: FOLLOWER
I20260806 22:49:07.499477 14928 consensus_queue.cc:260] T 1174265abf4a4444bcbef2c7a66a2be7 P 6001fbb534ca49bf960cc6d681df7958 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6001fbb534ca49bf960cc6d681df7958" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 40663 } } peers { permanent_uuid: "a0e72d89ce6943819b36930b465ba29d" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 37787 } } peers { permanent_uuid: "901eb508a51f4206bb28171ad7fcefde" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 44095 } }
I20260806 22:49:07.499809 14928 ts_tablet_manager.cc:1428] T 1174265abf4a4444bcbef2c7a66a2be7 P 6001fbb534ca49bf960cc6d681df7958: Time spent starting tablet: real 0.004s user 0.003s sys 0.001s
I20260806 22:49:07.499866 14906 heartbeater.cc:503] Master 127.18.45.254:39681 was elected leader, sending a full tablet report...
I20260806 22:49:07.500555 14927 raft_consensus.cc:348] T 1174265abf4a4444bcbef2c7a66a2be7 P a0e72d89ce6943819b36930b465ba29d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6001fbb534ca49bf960cc6d681df7958" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 40663 } } peers { permanent_uuid: "a0e72d89ce6943819b36930b465ba29d" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 37787 } } peers { permanent_uuid: "901eb508a51f4206bb28171ad7fcefde" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 44095 } }
I20260806 22:49:07.500733 14927 raft_consensus.cc:374] T 1174265abf4a4444bcbef2c7a66a2be7 P a0e72d89ce6943819b36930b465ba29d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:07.500770 14927 raft_consensus.cc:729] T 1174265abf4a4444bcbef2c7a66a2be7 P a0e72d89ce6943819b36930b465ba29d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0e72d89ce6943819b36930b465ba29d, State: Initialized, Role: FOLLOWER
I20260806 22:49:07.500898 14927 consensus_queue.cc:260] T 1174265abf4a4444bcbef2c7a66a2be7 P a0e72d89ce6943819b36930b465ba29d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6001fbb534ca49bf960cc6d681df7958" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 40663 } } peers { permanent_uuid: "a0e72d89ce6943819b36930b465ba29d" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 37787 } } peers { permanent_uuid: "901eb508a51f4206bb28171ad7fcefde" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 44095 } }
I20260806 22:49:07.501209 14927 ts_tablet_manager.cc:1428] T 1174265abf4a4444bcbef2c7a66a2be7 P a0e72d89ce6943819b36930b465ba29d: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260806 22:49:07.501374 14545 heartbeater.cc:503] Master 127.18.45.254:39681 was elected leader, sending a full tablet report...
I20260806 22:49:07.501516 14931 raft_consensus.cc:348] T 1174265abf4a4444bcbef2c7a66a2be7 P 901eb508a51f4206bb28171ad7fcefde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6001fbb534ca49bf960cc6d681df7958" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 40663 } } peers { permanent_uuid: "a0e72d89ce6943819b36930b465ba29d" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 37787 } } peers { permanent_uuid: "901eb508a51f4206bb28171ad7fcefde" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 44095 } }
I20260806 22:49:07.501672 14931 raft_consensus.cc:374] T 1174265abf4a4444bcbef2c7a66a2be7 P 901eb508a51f4206bb28171ad7fcefde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:07.501720 14931 raft_consensus.cc:729] T 1174265abf4a4444bcbef2c7a66a2be7 P 901eb508a51f4206bb28171ad7fcefde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 901eb508a51f4206bb28171ad7fcefde, State: Initialized, Role: FOLLOWER
I20260806 22:49:07.501854 14931 consensus_queue.cc:260] T 1174265abf4a4444bcbef2c7a66a2be7 P 901eb508a51f4206bb28171ad7fcefde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6001fbb534ca49bf960cc6d681df7958" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 40663 } } peers { permanent_uuid: "a0e72d89ce6943819b36930b465ba29d" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 37787 } } peers { permanent_uuid: "901eb508a51f4206bb28171ad7fcefde" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 44095 } }
I20260806 22:49:07.502173 14931 ts_tablet_manager.cc:1428] T 1174265abf4a4444bcbef2c7a66a2be7 P 901eb508a51f4206bb28171ad7fcefde: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260806 22:49:07.502228 14726 heartbeater.cc:503] Master 127.18.45.254:39681 was elected leader, sending a full tablet report...
I20260806 22:49:07.526911 14934 raft_consensus.cc:482] T 1174265abf4a4444bcbef2c7a66a2be7 P 6001fbb534ca49bf960cc6d681df7958 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 22:49:07.527171 14934 raft_consensus.cc:504] T 1174265abf4a4444bcbef2c7a66a2be7 P 6001fbb534ca49bf960cc6d681df7958 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6001fbb534ca49bf960cc6d681df7958" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 40663 } } peers { permanent_uuid: "a0e72d89ce6943819b36930b465ba29d" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 37787 } } peers { permanent_uuid: "901eb508a51f4206bb28171ad7fcefde" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 44095 } }
I20260806 22:49:07.527731 14934 leader_election.cc:288] T 1174265abf4a4444bcbef2c7a66a2be7 P 6001fbb534ca49bf960cc6d681df7958 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0e72d89ce6943819b36930b465ba29d (127.18.45.193:37787), 901eb508a51f4206bb28171ad7fcefde (127.18.45.194:44095)
I20260806 22:49:07.531773 14919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:07.528204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.195:44521 (local address 127.18.45.193:37787)
0806 22:49:07.528275 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:07.528277 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:07.528295 (+ 18us) server_negotiation.cc:407] Connection header received
0806 22:49:07.528336 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:07.528337 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:07.528373 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:07.528437 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:49:07.528689 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:07.529358 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:07.530766 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:07.531349 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:07.531442 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:07.531454 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:07.531640 (+ 186us) server_negotiation.cc:299] Negotiation successful
0806 22:49:07.531693 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 22:49:07.531772 14908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:07.528096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.193:37787 (local address 127.18.45.195:44521)
0806 22:49:07.528172 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:07.528187 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:07.528247 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:07.528466 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:07.528467 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:07.528470 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:49:07.528571 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:07.528573 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:07.529468 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:07.529469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:07.530673 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:07.530676 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:07.531454 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:07.531455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:07.531550 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:07.531554 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:49:07.531623 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 22:49:07.531677 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260806 22:49:07.532115 14926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:07.528540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.45.195:38323 (local address 127.18.45.194:44095)
0806 22:49:07.528629 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:07.528631 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:07.528642 (+ 11us) server_negotiation.cc:407] Connection header received
0806 22:49:07.528715 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:07.528716 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:07.528730 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:07.528809 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 22:49:07.529050 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:07.529758 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:07.530842 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:07.531572 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:07.531697 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:07.531708 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:07.531935 (+ 227us) server_negotiation.cc:299] Negotiation successful
0806 22:49:07.531998 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 22:49:07.532218 14484 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1174265abf4a4444bcbef2c7a66a2be7" candidate_uuid: "6001fbb534ca49bf960cc6d681df7958" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e72d89ce6943819b36930b465ba29d" is_pre_election: true
I20260806 22:49:07.532379 14938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:07.528264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.194:44095 (local address 127.18.45.195:38323)
0806 22:49:07.528547 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:07.528556 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:07.528629 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:07.528811 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:07.528812 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:07.528814 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 22:49:07.528953 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:07.528955 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:07.529824 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:07.529825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:07.530754 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:07.530757 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:07.531709 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:07.531711 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:07.531811 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:07.531815 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 22:49:07.532210 (+ 395us) client_negotiation.cc:240] Negotiation successful
0806 22:49:07.532267 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":185,"threads_started":1}
I20260806 22:49:07.532457 14484 raft_consensus.cc:2454] T 1174265abf4a4444bcbef2c7a66a2be7 P a0e72d89ce6943819b36930b465ba29d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6001fbb534ca49bf960cc6d681df7958 in term 0.
I20260806 22:49:07.532778 14753 leader_election.cc:302] T 1174265abf4a4444bcbef2c7a66a2be7 P 6001fbb534ca49bf960cc6d681df7958 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6001fbb534ca49bf960cc6d681df7958, a0e72d89ce6943819b36930b465ba29d; no voters:
I20260806 22:49:07.532923 14666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1174265abf4a4444bcbef2c7a66a2be7" candidate_uuid: "6001fbb534ca49bf960cc6d681df7958" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "901eb508a51f4206bb28171ad7fcefde" is_pre_election: true
I20260806 22:49:07.532987 14934 raft_consensus.cc:2781] T 1174265abf4a4444bcbef2c7a66a2be7 P 6001fbb534ca49bf960cc6d681df7958 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 22:49:07.533077 14934 raft_consensus.cc:482] T 1174265abf4a4444bcbef2c7a66a2be7 P 6001fbb534ca49bf960cc6d681df7958 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 22:49:07.533114 14934 raft_consensus.cc:3037] T 1174265abf4a4444bcbef2c7a66a2be7 P 6001fbb534ca49bf960cc6d681df7958 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:07.533108 14666 raft_consensus.cc:2454] T 1174265abf4a4444bcbef2c7a66a2be7 P 901eb508a51f4206bb28171ad7fcefde [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6001fbb534ca49bf960cc6d681df7958 in term 0.
I20260806 22:49:07.534080 14934 raft_consensus.cc:504] T 1174265abf4a4444bcbef2c7a66a2be7 P 6001fbb534ca49bf960cc6d681df7958 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6001fbb534ca49bf960cc6d681df7958" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 40663 } } peers { permanent_uuid: "a0e72d89ce6943819b36930b465ba29d" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 37787 } } peers { permanent_uuid: "901eb508a51f4206bb28171ad7fcefde" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 44095 } }
I20260806 22:49:07.534245 14934 leader_election.cc:288] T 1174265abf4a4444bcbef2c7a66a2be7 P 6001fbb534ca49bf960cc6d681df7958 [CANDIDATE]: Term 1 election: Requested vote from peers a0e72d89ce6943819b36930b465ba29d (127.18.45.193:37787), 901eb508a51f4206bb28171ad7fcefde (127.18.45.194:44095)
I20260806 22:49:07.534439 14484 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1174265abf4a4444bcbef2c7a66a2be7" candidate_uuid: "6001fbb534ca49bf960cc6d681df7958" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e72d89ce6943819b36930b465ba29d"
I20260806 22:49:07.534488 14666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1174265abf4a4444bcbef2c7a66a2be7" candidate_uuid: "6001fbb534ca49bf960cc6d681df7958" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "901eb508a51f4206bb28171ad7fcefde"
I20260806 22:49:07.534577 14666 raft_consensus.cc:3037] T 1174265abf4a4444bcbef2c7a66a2be7 P 901eb508a51f4206bb28171ad7fcefde [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:07.534578 14484 raft_consensus.cc:3037] T 1174265abf4a4444bcbef2c7a66a2be7 P a0e72d89ce6943819b36930b465ba29d [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:07.535833 14666 raft_consensus.cc:2454] T 1174265abf4a4444bcbef2c7a66a2be7 P 901eb508a51f4206bb28171ad7fcefde [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6001fbb534ca49bf960cc6d681df7958 in term 1.
I20260806 22:49:07.535833 14484 raft_consensus.cc:2454] T 1174265abf4a4444bcbef2c7a66a2be7 P a0e72d89ce6943819b36930b465ba29d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6001fbb534ca49bf960cc6d681df7958 in term 1.
I20260806 22:49:07.536080 14753 leader_election.cc:302] T 1174265abf4a4444bcbef2c7a66a2be7 P 6001fbb534ca49bf960cc6d681df7958 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6001fbb534ca49bf960cc6d681df7958, a0e72d89ce6943819b36930b465ba29d; no voters:
I20260806 22:49:07.536183 14934 raft_consensus.cc:2781] T 1174265abf4a4444bcbef2c7a66a2be7 P 6001fbb534ca49bf960cc6d681df7958 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:49:07.536391 14934 raft_consensus.cc:686] T 1174265abf4a4444bcbef2c7a66a2be7 P 6001fbb534ca49bf960cc6d681df7958 [term 1 LEADER]: Becoming Leader. State: Replica: 6001fbb534ca49bf960cc6d681df7958, State: Running, Role: LEADER
I20260806 22:49:07.536527 14934 consensus_queue.cc:237] T 1174265abf4a4444bcbef2c7a66a2be7 P 6001fbb534ca49bf960cc6d681df7958 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6001fbb534ca49bf960cc6d681df7958" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 40663 } } peers { permanent_uuid: "a0e72d89ce6943819b36930b465ba29d" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 37787 } } peers { permanent_uuid: "901eb508a51f4206bb28171ad7fcefde" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 44095 } }
I20260806 22:49:07.537706 14287 catalog_manager.cc:5495] T 1174265abf4a4444bcbef2c7a66a2be7 P 6001fbb534ca49bf960cc6d681df7958 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6001fbb534ca49bf960cc6d681df7958 (127.18.45.195). New cstate: current_term: 1 leader_uuid: "6001fbb534ca49bf960cc6d681df7958" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6001fbb534ca49bf960cc6d681df7958" member_type: VOTER last_known_addr { host: "127.18.45.195" port: 40663 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e72d89ce6943819b36930b465ba29d" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 37787 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "901eb508a51f4206bb28171ad7fcefde" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 44095 } health_report { overall_health: UNKNOWN } } }
I20260806 22:49:07.548027 14920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:07.544671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47888 (local address 127.18.45.195:40663)
0806 22:49:07.544761 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:07.544763 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:07.544773 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:49:07.544852 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:07.544853 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:07.544877 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:07.544987 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:49:07.545209 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:07.545820 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:07.546749 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:07.547140 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:07.547200 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:07.547443 (+ 243us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:49:07.547585 (+ 142us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:49:07.547670 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:07.547757 (+ 87us) server_negotiation.cc:299] Negotiation successful
0806 22:49:07.547781 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 22:49:07.550840 14666 raft_consensus.cc:1261] T 1174265abf4a4444bcbef2c7a66a2be7 P 901eb508a51f4206bb28171ad7fcefde [term 1 FOLLOWER]: Refusing update from remote peer 6001fbb534ca49bf960cc6d681df7958: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:49:07.550840 14484 raft_consensus.cc:1261] T 1174265abf4a4444bcbef2c7a66a2be7 P a0e72d89ce6943819b36930b465ba29d [term 1 FOLLOWER]: Refusing update from remote peer 6001fbb534ca49bf960cc6d681df7958: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:49:07.551432 14939 consensus_queue.cc:1035] T 1174265abf4a4444bcbef2c7a66a2be7 P 6001fbb534ca49bf960cc6d681df7958 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e72d89ce6943819b36930b465ba29d" member_type: VOTER last_known_addr { host: "127.18.45.193" port: 37787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:49:07.551669 14934 consensus_queue.cc:1035] T 1174265abf4a4444bcbef2c7a66a2be7 P 6001fbb534ca49bf960cc6d681df7958 [LEADER]: Connected to new peer: Peer: permanent_uuid: "901eb508a51f4206bb28171ad7fcefde" member_type: VOTER last_known_addr { host: "127.18.45.194" port: 44095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 22:49:07.554201 14728 tablet.cc:2369] T 1174265abf4a4444bcbef2c7a66a2be7 P 901eb508a51f4206bb28171ad7fcefde: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 22:49:07.556715 14944 mvcc.cc:205] Tried to move back new op lower bound from 7315687618763915264 to 7315687618710511616. Current Snapshot: MvccSnapshot[applied={T|T < 7315687618763915264}]
I20260806 22:49:07.557400 14942 mvcc.cc:205] Tried to move back new op lower bound from 7315687618763915264 to 7315687618710511616. Current Snapshot: MvccSnapshot[applied={T|T < 7315687618763915264}]
I20260806 22:49:07.560127 14287 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48948:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:49:07.560264 14287 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48948:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 22:49:07.561471 14287 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5d9eb43716d34e02b5769d683bdca833: Sending DeleteTablet for 3 replicas of tablet 1174265abf4a4444bcbef2c7a66a2be7
I20260806 22:49:07.561967 14458 tablet_service.cc:1514] Processing DeleteTablet for tablet 1174265abf4a4444bcbef2c7a66a2be7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:49:07 UTC) from {username='slave'} at 127.0.0.1:41406
I20260806 22:49:07.562088 14818 tablet_service.cc:1514] Processing DeleteTablet for tablet 1174265abf4a4444bcbef2c7a66a2be7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:49:07 UTC) from {username='slave'} at 127.0.0.1:47878
I20260806 22:49:07.562307 14636 tablet_service.cc:1514] Processing DeleteTablet for tablet 1174265abf4a4444bcbef2c7a66a2be7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 22:49:07 UTC) from {username='slave'} at 127.0.0.1:34234
I20260806 22:49:07.562383 14956 tablet_replica.cc:330] T 1174265abf4a4444bcbef2c7a66a2be7 P 6001fbb534ca49bf960cc6d681df7958: stopping tablet replica
I20260806 22:49:07.562515 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 14373
I20260806 22:49:07.562767 14956 raft_consensus.cc:2229] T 1174265abf4a4444bcbef2c7a66a2be7 P 6001fbb534ca49bf960cc6d681df7958 [term 1 LEADER]: Raft consensus shutting down.
I20260806 22:49:07.572933 14956 raft_consensus.cc:2258] T 1174265abf4a4444bcbef2c7a66a2be7 P 6001fbb534ca49bf960cc6d681df7958 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:49:07.573457 14958 tablet_replica.cc:330] T 1174265abf4a4444bcbef2c7a66a2be7 P 901eb508a51f4206bb28171ad7fcefde: stopping tablet replica
I20260806 22:49:07.573698 14958 raft_consensus.cc:2229] T 1174265abf4a4444bcbef2c7a66a2be7 P 901eb508a51f4206bb28171ad7fcefde [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 22:49:07.573848 14958 raft_consensus.cc:2258] T 1174265abf4a4444bcbef2c7a66a2be7 P 901eb508a51f4206bb28171ad7fcefde [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 22:49:07.574381 14956 ts_tablet_manager.cc:1905] T 1174265abf4a4444bcbef2c7a66a2be7 P 6001fbb534ca49bf960cc6d681df7958: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:49:07.574537 14958 ts_tablet_manager.cc:1905] T 1174265abf4a4444bcbef2c7a66a2be7 P 901eb508a51f4206bb28171ad7fcefde: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 22:49:07.576638 14956 ts_tablet_manager.cc:1918] T 1174265abf4a4444bcbef2c7a66a2be7 P 6001fbb534ca49bf960cc6d681df7958: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:49:07.576747 14956 log.cc:1196] T 1174265abf4a4444bcbef2c7a66a2be7 P 6001fbb534ca49bf960cc6d681df7958: Deleting WAL directory at /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056275803658-18615-0/minicluster-data/ts-2/wal/wals/1174265abf4a4444bcbef2c7a66a2be7
I20260806 22:49:07.576889 14958 ts_tablet_manager.cc:1918] T 1174265abf4a4444bcbef2c7a66a2be7 P 901eb508a51f4206bb28171ad7fcefde: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 22:49:07.577057 14958 log.cc:1196] T 1174265abf4a4444bcbef2c7a66a2be7 P 901eb508a51f4206bb28171ad7fcefde: Deleting WAL directory at /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786056275803658-18615-0/minicluster-data/ts-1/wal/wals/1174265abf4a4444bcbef2c7a66a2be7
I20260806 22:49:07.577092 14956 ts_tablet_manager.cc:1939] T 1174265abf4a4444bcbef2c7a66a2be7 P 6001fbb534ca49bf960cc6d681df7958: Deleting consensus metadata
I20260806 22:49:07.577431 14958 ts_tablet_manager.cc:1939] T 1174265abf4a4444bcbef2c7a66a2be7 P 901eb508a51f4206bb28171ad7fcefde: Deleting consensus metadata
I20260806 22:49:07.577687 14272 catalog_manager.cc:4841] TS 6001fbb534ca49bf960cc6d681df7958 (127.18.45.195:40663): tablet 1174265abf4a4444bcbef2c7a66a2be7 (table test-table [id=ab65a706a44d4b008026bcb542f4460b]) successfully deleted
I20260806 22:49:07.578063 14270 catalog_manager.cc:4841] TS 901eb508a51f4206bb28171ad7fcefde (127.18.45.194:44095): tablet 1174265abf4a4444bcbef2c7a66a2be7 (table test-table [id=ab65a706a44d4b008026bcb542f4460b]) successfully deleted
W20260806 22:49:07.584846 14270 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.45.193:37787 (error 108)
I20260806 22:49:07.585120 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 14548
I20260806 22:49:07.591692 14917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:07.591477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.193:37787 (local address 127.0.0.1:41420)
0806 22:49:07.591558 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:07.591627 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.193:37787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260806 22:49:07.591862 14270 catalog_manager.cc:4584] TS a0e72d89ce6943819b36930b465ba29d (127.18.45.193:37787): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1174265abf4a4444bcbef2c7a66a2be7: Network error: Client connection negotiation failed: client connection to 127.18.45.193:37787: connect: Connection refused (error 111)
I20260806 22:49:07.603499 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 14731
I20260806 22:49:07.619342 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 14247
2026-08-06T22:49:07Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (840 ms)
[----------] 6 tests from EncryptionPolicyTest (10291 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056275803658-18615-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:49:07 dist-test-slave-h3ft krb5kdc[14965](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:49:07 dist-test-slave-h3ft krb5kdc[14965](info): set up 2 sockets
Aug 06 22:49:07 dist-test-slave-h3ft krb5kdc[14965](info): commencing operation
krb5kdc: starting...
W20260806 22:49:08.892731 18615 mini_kdc.cc:121] Time spent starting KDC: real 1.243s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Aug 06 22:49:08 dist-test-slave-h3ft krb5kdc[14965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056548, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:49:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:49:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056275803658-18615-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056275803658-18615-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:49:09.046649 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056275803658-18615-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056275803658-18615-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056275803658-18615-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43781
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056275803658-18615-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43781
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:49:09.148701 14981 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:09.149003 14981 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:09.149041 14981 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:09.150306 14981 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:49:09.150378 14981 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:09.150403 14981 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:09.150439 14981 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:49:09.150489 14981 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:49:09.152263 14981 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056275803658-18615-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056275803658-18615-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43781
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056275803658-18615-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43781
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14981
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056275803658-18615-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:49:09.152627 14981 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:09.152913 14981 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:09.158741 14991 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:09.158747 14989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:09.159118 14981 server_base.cc:1034] running on GCE node
W20260806 22:49:09.158816 14987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:09.159544 14981 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:09.159986 14981 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:09.161193 14981 hybrid_clock.cc:648] HybridClock initialized: now 1786056549161170 us; error 52 us; skew 500 ppm
Aug 06 22:49:09 dist-test-slave-h3ft krb5kdc[14965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056549, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:09.164438 14981 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:49:09.165014 14981 webserver.cc:460] Webserver started at http://127.0.0.1:37779/ using document root <none> and password file <none>
I20260806 22:49:09.165303 14981 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:09.165382 14981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:09.165553 14981 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:49:09.166890 14981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056275803658-18615-0/minicluster-data/master-0/data/instance:
uuid: "0f8366d84c1747aca647dca07ee6db51"
format_stamp: "Formatted at 2026-08-06 22:49:09 on dist-test-slave-h3ft"
server_key: "7b215a8a33b0dc2b0baa1b715e0ec0d5"
server_key_iv: "d928ac3ec843ece3e20693b446757339"
server_key_version: "encryptionkey@0"
I20260806 22:49:09.167382 14981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056275803658-18615-0/minicluster-data/master-0/wal/instance:
uuid: "0f8366d84c1747aca647dca07ee6db51"
format_stamp: "Formatted at 2026-08-06 22:49:09 on dist-test-slave-h3ft"
server_key: "7b215a8a33b0dc2b0baa1b715e0ec0d5"
server_key_iv: "d928ac3ec843ece3e20693b446757339"
server_key_version: "encryptionkey@0"
I20260806 22:49:09.168998 14981 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:49:09.169912 15000 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:09.170212 14981 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:49:09.170370 14981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056275803658-18615-0/minicluster-data/master-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056275803658-18615-0/minicluster-data/master-0/wal
uuid: "0f8366d84c1747aca647dca07ee6db51"
format_stamp: "Formatted at 2026-08-06 22:49:09 on dist-test-slave-h3ft"
server_key: "7b215a8a33b0dc2b0baa1b715e0ec0d5"
server_key_iv: "d928ac3ec843ece3e20693b446757339"
server_key_version: "encryptionkey@0"
I20260806 22:49:09.170472 14981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056275803658-18615-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056275803658-18615-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056275803658-18615-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:09.187551 14981 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:09.190372 14981 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:09.190585 14981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:09.195977 14981 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43781
I20260806 22:49:09.196005 15077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43781 every 8 connection(s)
I20260806 22:49:09.196470 14981 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
I20260806 22:49:09.196696 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 14981
I20260806 22:49:09.196833 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786056275803658-18615-0/minicluster-data/master-0/wal/instance
I20260806 22:49:09.197119 18615 external_mini_cluster.cc:1428] Setting key 510b70a0199af6012180315b7424eaff
I20260806 22:49:09.197773 15079 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:09.201588 15079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f8366d84c1747aca647dca07ee6db51: Bootstrap starting.
I20260806 22:49:09.202472 15079 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0f8366d84c1747aca647dca07ee6db51: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:09.202862 15079 log.cc:824] T 00000000000000000000000000000000 P 0f8366d84c1747aca647dca07ee6db51: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:09.203867 15079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f8366d84c1747aca647dca07ee6db51: No bootstrap required, opened a new log
Aug 06 22:49:09 dist-test-slave-h3ft krb5kdc[14965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056548, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 22:49:09.206732 15079 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0f8366d84c1747aca647dca07ee6db51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f8366d84c1747aca647dca07ee6db51" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43781 } }
I20260806 22:49:09.206950 15079 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0f8366d84c1747aca647dca07ee6db51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:09.207023 15079 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0f8366d84c1747aca647dca07ee6db51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f8366d84c1747aca647dca07ee6db51, State: Initialized, Role: FOLLOWER
I20260806 22:49:09.207237 15079 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0f8366d84c1747aca647dca07ee6db51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f8366d84c1747aca647dca07ee6db51" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43781 } }
I20260806 22:49:09.207346 15079 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0f8366d84c1747aca647dca07ee6db51 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:49:09.207389 15079 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0f8366d84c1747aca647dca07ee6db51 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:49:09.207453 15079 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0f8366d84c1747aca647dca07ee6db51 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:09.208309 15079 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0f8366d84c1747aca647dca07ee6db51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f8366d84c1747aca647dca07ee6db51" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43781 } }
I20260806 22:49:09.208439 15079 leader_election.cc:302] T 00000000000000000000000000000000 P 0f8366d84c1747aca647dca07ee6db51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0f8366d84c1747aca647dca07ee6db51; no voters:
I20260806 22:49:09.208674 15079 leader_election.cc:288] T 00000000000000000000000000000000 P 0f8366d84c1747aca647dca07ee6db51 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:49:09.208702 15088 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0f8366d84c1747aca647dca07ee6db51 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:49:09.208835 15088 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0f8366d84c1747aca647dca07ee6db51 [term 1 LEADER]: Becoming Leader. State: Replica: 0f8366d84c1747aca647dca07ee6db51, State: Running, Role: LEADER
I20260806 22:49:09.209010 15079 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0f8366d84c1747aca647dca07ee6db51 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:49:09.209029 15088 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0f8366d84c1747aca647dca07ee6db51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f8366d84c1747aca647dca07ee6db51" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43781 } }
I20260806 22:49:09.209551 15091 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f8366d84c1747aca647dca07ee6db51 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0f8366d84c1747aca647dca07ee6db51. Latest consensus state: current_term: 1 leader_uuid: "0f8366d84c1747aca647dca07ee6db51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f8366d84c1747aca647dca07ee6db51" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43781 } } }
I20260806 22:49:09.209656 15091 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f8366d84c1747aca647dca07ee6db51 [sys.catalog]: This master's current role is: LEADER
I20260806 22:49:09.209935 15090 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f8366d84c1747aca647dca07ee6db51 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0f8366d84c1747aca647dca07ee6db51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f8366d84c1747aca647dca07ee6db51" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43781 } } }
I20260806 22:49:09.210429 15090 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f8366d84c1747aca647dca07ee6db51 [sys.catalog]: This master's current role is: LEADER
I20260806 22:49:09.210489 15100 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:49:09.210748 15085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:09.197826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41289 (local address 127.0.0.1:43781)
0806 22:49:09.198050 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:09.198061 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:09.198099 (+ 38us) server_negotiation.cc:407] Connection header received
0806 22:49:09.198721 (+ 622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:09.198732 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:09.198809 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:09.199024 (+ 215us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:09.199762 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:09.201243 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:09.202051 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:09.202734 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:09.202856 (+ 122us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:09.206033 (+ 3177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:09.206040 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:09.206040 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:09.206057 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:09.208317 (+ 2260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:09.208855 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:09.208858 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:09.208859 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:09.208993 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:09.209251 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:09.209251 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:09.209251 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:09.209587 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:09.209744 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:09.210335 (+ 591us) server_negotiation.cc:299] Negotiation successful
0806 22:49:09.210461 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":113,"threads_started":1}
I20260806 22:49:09.211241 15100 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:49:09.212134 18615 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:49:09.213044 15100 catalog_manager.cc:1347] Generated new cluster ID: 3a56a0c088ae48d1a7525bbe0845d35f
I20260806 22:49:09.213101 15100 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:49:09.221915 15085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:09.215132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51671 (local address 127.0.0.1:43781)
0806 22:49:09.215174 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:09.215176 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:09.215188 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:49:09.215213 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:09.215214 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:09.215225 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:09.215286 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:09.215891 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:09.216426 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:09.217154 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:09.217391 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:09.217472 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:09.218858 (+ 1386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:09.218860 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:09.218860 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:09.218868 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:09.220543 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:09.220944 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:09.220945 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:09.220945 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:09.221012 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:09.221250 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:09.221252 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:09.221252 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:09.221378 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:09.221531 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:09.221635 (+ 104us) server_negotiation.cc:299] Negotiation successful
0806 22:49:09.221703 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:49:09.232954 15100 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:49:09.233876 15100 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:49:09.254726 15100 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0f8366d84c1747aca647dca07ee6db51: Generated new TSK 0
I20260806 22:49:09.255098 15100 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:49:09.262929 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 14981
W20260806 22:49:09.279969 14960 connection.cc:535] client connection to 127.0.0.1:43781 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43781): Connection reset by peer (error 104)
2026-08-06T22:49:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1644 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056275803658-18615-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:49:09 dist-test-slave-h3ft krb5kdc[15124](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:49:09 dist-test-slave-h3ft krb5kdc[15124](info): set up 2 sockets
Aug 06 22:49:09 dist-test-slave-h3ft krb5kdc[15124](info): commencing operation
krb5kdc: starting...
W20260806 22:49:10.550034 18615 mini_kdc.cc:121] Time spent starting KDC: real 1.256s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Aug 06 22:49:10 dist-test-slave-h3ft krb5kdc[15124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056550, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:49:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:49:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056275803658-18615-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056275803658-18615-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:49:10.695969 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056275803658-18615-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056275803658-18615-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056275803658-18615-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44671
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056275803658-18615-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44671
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:49:10.796047 15147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:10.796358 15147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:10.796403 15147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:10.797724 15147 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:49:10.797806 15147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:10.797840 15147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:10.797873 15147 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:49:10.797904 15147 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:49:10.799691 15147 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056275803658-18615-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056275803658-18615-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44671
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056275803658-18615-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44671
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15147
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056275803658-18615-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:49:10.800052 15147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:10.800357 15147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:10.806416 15154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:10.806495 15153 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:10.806465 15156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:10.806967 15147 server_base.cc:1034] running on GCE node
I20260806 22:49:10.807379 15147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:10.807775 15147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:10.808986 15147 hybrid_clock.cc:648] HybridClock initialized: now 1786056550808964 us; error 47 us; skew 500 ppm
Aug 06 22:49:10 dist-test-slave-h3ft krb5kdc[15124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056550, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:10.812407 15147 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:49:10.813083 15147 webserver.cc:460] Webserver started at http://127.0.0.1:37279/ using document root <none> and password file <none>
I20260806 22:49:10.813421 15147 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:10.813504 15147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:10.813695 15147 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:49:10.815207 15147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056275803658-18615-0/minicluster-data/master-0/data/instance:
uuid: "e7d11818bad24c3ea6e07605991bf964"
format_stamp: "Formatted at 2026-08-06 22:49:10 on dist-test-slave-h3ft"
server_key: "9842609366953467f3c3e79c1e801f3d"
server_key_iv: "857dfdd69c8e6875871716ba34a8aea1"
server_key_version: "encryptionkey@0"
I20260806 22:49:10.815667 15147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056275803658-18615-0/minicluster-data/master-0/wal/instance:
uuid: "e7d11818bad24c3ea6e07605991bf964"
format_stamp: "Formatted at 2026-08-06 22:49:10 on dist-test-slave-h3ft"
server_key: "9842609366953467f3c3e79c1e801f3d"
server_key_iv: "857dfdd69c8e6875871716ba34a8aea1"
server_key_version: "encryptionkey@0"
I20260806 22:49:10.817435 15147 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 22:49:10.818461 15164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:10.818704 15147 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:49:10.818835 15147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056275803658-18615-0/minicluster-data/master-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056275803658-18615-0/minicluster-data/master-0/wal
uuid: "e7d11818bad24c3ea6e07605991bf964"
format_stamp: "Formatted at 2026-08-06 22:49:10 on dist-test-slave-h3ft"
server_key: "9842609366953467f3c3e79c1e801f3d"
server_key_iv: "857dfdd69c8e6875871716ba34a8aea1"
server_key_version: "encryptionkey@0"
I20260806 22:49:10.818929 15147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056275803658-18615-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056275803658-18615-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056275803658-18615-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:10.837987 15147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:10.840562 15147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:10.840791 15147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:10.847245 15147 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44671
I20260806 22:49:10.847290 15233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44671 every 8 connection(s)
I20260806 22:49:10.847842 15147 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
I20260806 22:49:10.849239 15234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:10.853708 15234 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7d11818bad24c3ea6e07605991bf964: Bootstrap starting.
I20260806 22:49:10.854230 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 15147
I20260806 22:49:10.854385 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786056275803658-18615-0/minicluster-data/master-0/wal/instance
I20260806 22:49:10.854645 18615 external_mini_cluster.cc:1428] Setting key b2684ab94cbf1e4dd9e9cdb634aa3517
I20260806 22:49:10.854854 15234 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e7d11818bad24c3ea6e07605991bf964: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:10.855396 15234 log.cc:824] T 00000000000000000000000000000000 P e7d11818bad24c3ea6e07605991bf964: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:10.856350 15234 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7d11818bad24c3ea6e07605991bf964: No bootstrap required, opened a new log
I20260806 22:49:10.859123 15234 raft_consensus.cc:348] T 00000000000000000000000000000000 P e7d11818bad24c3ea6e07605991bf964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7d11818bad24c3ea6e07605991bf964" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44671 } }
I20260806 22:49:10.859349 15234 raft_consensus.cc:374] T 00000000000000000000000000000000 P e7d11818bad24c3ea6e07605991bf964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:10.859395 15234 raft_consensus.cc:729] T 00000000000000000000000000000000 P e7d11818bad24c3ea6e07605991bf964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7d11818bad24c3ea6e07605991bf964, State: Initialized, Role: FOLLOWER
I20260806 22:49:10.859582 15234 consensus_queue.cc:260] T 00000000000000000000000000000000 P e7d11818bad24c3ea6e07605991bf964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7d11818bad24c3ea6e07605991bf964" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44671 } }
I20260806 22:49:10.859686 15234 raft_consensus.cc:388] T 00000000000000000000000000000000 P e7d11818bad24c3ea6e07605991bf964 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:49:10.859720 15234 raft_consensus.cc:482] T 00000000000000000000000000000000 P e7d11818bad24c3ea6e07605991bf964 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:49:10.859774 15234 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e7d11818bad24c3ea6e07605991bf964 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:10.860855 15234 raft_consensus.cc:504] T 00000000000000000000000000000000 P e7d11818bad24c3ea6e07605991bf964 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7d11818bad24c3ea6e07605991bf964" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44671 } }
I20260806 22:49:10.861014 15234 leader_election.cc:302] T 00000000000000000000000000000000 P e7d11818bad24c3ea6e07605991bf964 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e7d11818bad24c3ea6e07605991bf964; no voters:
I20260806 22:49:10.861315 15234 leader_election.cc:288] T 00000000000000000000000000000000 P e7d11818bad24c3ea6e07605991bf964 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:49:10.861344 15241 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e7d11818bad24c3ea6e07605991bf964 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:49:10.861500 15241 raft_consensus.cc:686] T 00000000000000000000000000000000 P e7d11818bad24c3ea6e07605991bf964 [term 1 LEADER]: Becoming Leader. State: Replica: e7d11818bad24c3ea6e07605991bf964, State: Running, Role: LEADER
I20260806 22:49:10.861608 15241 consensus_queue.cc:237] T 00000000000000000000000000000000 P e7d11818bad24c3ea6e07605991bf964 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7d11818bad24c3ea6e07605991bf964" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44671 } }
I20260806 22:49:10.861723 15234 sys_catalog.cc:564] T 00000000000000000000000000000000 P e7d11818bad24c3ea6e07605991bf964 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:49:10.862010 15241 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7d11818bad24c3ea6e07605991bf964 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e7d11818bad24c3ea6e07605991bf964. Latest consensus state: current_term: 1 leader_uuid: "e7d11818bad24c3ea6e07605991bf964" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7d11818bad24c3ea6e07605991bf964" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44671 } } }
I20260806 22:49:10.862093 15241 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7d11818bad24c3ea6e07605991bf964 [sys.catalog]: This master's current role is: LEADER
I20260806 22:49:10.862234 15242 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7d11818bad24c3ea6e07605991bf964 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e7d11818bad24c3ea6e07605991bf964" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7d11818bad24c3ea6e07605991bf964" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44671 } } }
I20260806 22:49:10.862308 15239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:10.855402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54355 (local address 127.0.0.1:44671)
0806 22:49:10.855656 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:10.855665 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:10.855725 (+ 60us) server_negotiation.cc:407] Connection header received
0806 22:49:10.856410 (+ 685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:10.856422 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:10.856502 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:10.856666 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:10.857022 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:10.858100 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:10.859491 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:10.860226 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:10.860388 (+ 162us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:10.861353 (+ 965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:10.861357 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:10.861358 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:49:10.861655 (+ 297us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:10.861705 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:49:10.861723 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:10.861869 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:10.861976 (+ 107us) server_negotiation.cc:299] Negotiation successful
0806 22:49:10.862011 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":113,"threads_started":1}
I20260806 22:49:10.862762 15242 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7d11818bad24c3ea6e07605991bf964 [sys.catalog]: This master's current role is: LEADER
I20260806 22:49:10.863617 18615 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:49:10.863593 15246 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:49:10.864346 15246 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:49:10.866515 15246 catalog_manager.cc:1347] Generated new cluster ID: 9db549ccbaae40d6be4d6673c2136c37
I20260806 22:49:10.866575 15246 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:49:10.869824 15239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:10.865718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44599 (local address 127.0.0.1:44671)
0806 22:49:10.865762 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:10.865764 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:10.865957 (+ 193us) server_negotiation.cc:407] Connection header received
0806 22:49:10.865985 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:10.865986 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:10.865999 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:10.866070 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:10.866241 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:10.867051 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:10.868214 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:10.868553 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:10.868635 (+ 82us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:10.869530 (+ 895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:10.869532 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:10.869532 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:49:10.869537 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:10.869558 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:49:10.869563 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:10.869638 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:10.869720 (+ 82us) server_negotiation.cc:299] Negotiation successful
0806 22:49:10.869735 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:49:10.875687 15246 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:49:10.876420 15246 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:49:10.880992 15246 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e7d11818bad24c3ea6e07605991bf964: Generated new TSK 0
I20260806 22:49:10.881253 15246 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:49:10.884646 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 15147
W20260806 22:49:10.900122 15122 connection.cc:535] client connection to 127.0.0.1:44671 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44671): Connection reset by peer (error 104)
2026-08-06T22:49:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1620 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056275803658-18615-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:49:10 dist-test-slave-h3ft krb5kdc[15281](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:49:10 dist-test-slave-h3ft krb5kdc[15281](info): set up 2 sockets
Aug 06 22:49:10 dist-test-slave-h3ft krb5kdc[15281](info): commencing operation
krb5kdc: starting...
W20260806 22:49:12.201020 18615 mini_kdc.cc:121] Time spent starting KDC: real 1.286s user 0.002s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Aug 06 22:49:12 dist-test-slave-h3ft krb5kdc[15281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056552, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:49:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:49:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056275803658-18615-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056275803658-18615-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:49:12.346956 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056275803658-18615-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056275803658-18615-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056275803658-18615-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33227
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056275803658-18615-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33227
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:49:12.449678 15297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:12.449997 15297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:12.450039 15297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:12.452574 15297 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:49:12.452646 15297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:12.452674 15297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:12.452698 15297 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:49:12.452728 15297 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:49:12.455255 15297 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056275803658-18615-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056275803658-18615-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33227
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056275803658-18615-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33227
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15297
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056275803658-18615-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:49:12.455610 15297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:12.455920 15297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:12.462423 15309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:12.462440 15307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:12.462430 15304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:12.467271 15297 server_base.cc:1034] running on GCE node
I20260806 22:49:12.467687 15297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:12.468173 15297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:12.469386 15297 hybrid_clock.cc:648] HybridClock initialized: now 1786056552469356 us; error 47 us; skew 500 ppm
Aug 06 22:49:12 dist-test-slave-h3ft krb5kdc[15281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056552, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:12.473430 15297 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:49:12.474248 15297 webserver.cc:460] Webserver started at http://127.0.0.1:32977/ using document root <none> and password file <none>
I20260806 22:49:12.474630 15297 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:12.474710 15297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:12.474915 15297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:49:12.477150 15297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056275803658-18615-0/minicluster-data/master-0/data/instance:
uuid: "eb89ead419774f0ebac1be475b98d199"
format_stamp: "Formatted at 2026-08-06 22:49:12 on dist-test-slave-h3ft"
server_key: "0befb0bc0fb575d516131dc917a71af4"
server_key_iv: "ed775ce4a1a7e1b7a695bdccafde444a"
server_key_version: "encryptionkey@0"
I20260806 22:49:12.477730 15297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056275803658-18615-0/minicluster-data/master-0/wal/instance:
uuid: "eb89ead419774f0ebac1be475b98d199"
format_stamp: "Formatted at 2026-08-06 22:49:12 on dist-test-slave-h3ft"
server_key: "0befb0bc0fb575d516131dc917a71af4"
server_key_iv: "ed775ce4a1a7e1b7a695bdccafde444a"
server_key_version: "encryptionkey@0"
I20260806 22:49:12.480228 15297 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.005s sys 0.000s
I20260806 22:49:12.481383 15321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:12.481572 15297 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:49:12.481679 15297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056275803658-18615-0/minicluster-data/master-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056275803658-18615-0/minicluster-data/master-0/wal
uuid: "eb89ead419774f0ebac1be475b98d199"
format_stamp: "Formatted at 2026-08-06 22:49:12 on dist-test-slave-h3ft"
server_key: "0befb0bc0fb575d516131dc917a71af4"
server_key_iv: "ed775ce4a1a7e1b7a695bdccafde444a"
server_key_version: "encryptionkey@0"
I20260806 22:49:12.481781 15297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056275803658-18615-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056275803658-18615-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056275803658-18615-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:12.487658 15297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:12.490805 15297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:12.490986 15297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:12.497263 15297 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33227
I20260806 22:49:12.497325 15384 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33227 every 8 connection(s)
I20260806 22:49:12.497730 15297 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
I20260806 22:49:12.499151 15386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:12.501959 15386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb89ead419774f0ebac1be475b98d199: Bootstrap starting.
I20260806 22:49:12.502722 15386 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb89ead419774f0ebac1be475b98d199: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:12.503161 15386 log.cc:824] T 00000000000000000000000000000000 P eb89ead419774f0ebac1be475b98d199: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:12.504045 15386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb89ead419774f0ebac1be475b98d199: No bootstrap required, opened a new log
I20260806 22:49:12.505944 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 15297
I20260806 22:49:12.506054 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786056275803658-18615-0/minicluster-data/master-0/wal/instance
I20260806 22:49:12.505936 15386 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb89ead419774f0ebac1be475b98d199 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb89ead419774f0ebac1be475b98d199" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33227 } }
I20260806 22:49:12.506202 15386 raft_consensus.cc:374] T 00000000000000000000000000000000 P eb89ead419774f0ebac1be475b98d199 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:12.506280 15386 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb89ead419774f0ebac1be475b98d199 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb89ead419774f0ebac1be475b98d199, State: Initialized, Role: FOLLOWER
I20260806 22:49:12.506340 18615 external_mini_cluster.cc:1428] Setting key 21c59a96259f5fff3c3937e33d8d30de
I20260806 22:49:12.506471 15386 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb89ead419774f0ebac1be475b98d199 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb89ead419774f0ebac1be475b98d199" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33227 } }
I20260806 22:49:12.506568 15386 raft_consensus.cc:388] T 00000000000000000000000000000000 P eb89ead419774f0ebac1be475b98d199 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:49:12.506608 15386 raft_consensus.cc:482] T 00000000000000000000000000000000 P eb89ead419774f0ebac1be475b98d199 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:49:12.506660 15386 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb89ead419774f0ebac1be475b98d199 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:12.507506 15386 raft_consensus.cc:504] T 00000000000000000000000000000000 P eb89ead419774f0ebac1be475b98d199 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb89ead419774f0ebac1be475b98d199" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33227 } }
I20260806 22:49:12.507645 15386 leader_election.cc:302] T 00000000000000000000000000000000 P eb89ead419774f0ebac1be475b98d199 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eb89ead419774f0ebac1be475b98d199; no voters:
I20260806 22:49:12.507838 15386 leader_election.cc:288] T 00000000000000000000000000000000 P eb89ead419774f0ebac1be475b98d199 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:49:12.507872 15392 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eb89ead419774f0ebac1be475b98d199 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:49:12.508044 15392 raft_consensus.cc:686] T 00000000000000000000000000000000 P eb89ead419774f0ebac1be475b98d199 [term 1 LEADER]: Becoming Leader. State: Replica: eb89ead419774f0ebac1be475b98d199, State: Running, Role: LEADER
I20260806 22:49:12.508144 15386 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb89ead419774f0ebac1be475b98d199 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:49:12.508149 15392 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb89ead419774f0ebac1be475b98d199 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb89ead419774f0ebac1be475b98d199" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33227 } }
I20260806 22:49:12.508678 15392 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb89ead419774f0ebac1be475b98d199 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb89ead419774f0ebac1be475b98d199. Latest consensus state: current_term: 1 leader_uuid: "eb89ead419774f0ebac1be475b98d199" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb89ead419774f0ebac1be475b98d199" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33227 } } }
I20260806 22:49:12.508760 15392 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb89ead419774f0ebac1be475b98d199 [sys.catalog]: This master's current role is: LEADER
I20260806 22:49:12.509002 15394 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb89ead419774f0ebac1be475b98d199 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eb89ead419774f0ebac1be475b98d199" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb89ead419774f0ebac1be475b98d199" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33227 } } }
I20260806 22:49:12.509083 15394 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb89ead419774f0ebac1be475b98d199 [sys.catalog]: This master's current role is: LEADER
I20260806 22:49:12.509859 15391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:12.507071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51383 (local address 127.0.0.1:33227)
0806 22:49:12.507770 (+ 699us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:12.507788 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:12.507830 (+ 42us) server_negotiation.cc:407] Connection header received
0806 22:49:12.507991 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:12.508001 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:12.508038 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:12.508559 (+ 521us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:12.509165 (+ 606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:12.509170 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:12.509170 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:49:12.509175 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:12.509220 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:49:12.509235 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:12.509304 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:12.509563 (+ 259us) server_negotiation.cc:299] Negotiation successful
0806 22:49:12.509627 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":629,"thread_start_us":580,"threads_started":1}
W20260806 22:49:12.510620 15413 catalog_manager.cc:1558] T 00000000000000000000000000000000 P eb89ead419774f0ebac1be475b98d199: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:49:12.510694 15413 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:49:12.510762 15399 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:49:12.510942 15399 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:49:12.511126 18615 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:49:12.513285 15399 catalog_manager.cc:1347] Generated new cluster ID: 813c3e387ba243929ba82802db00c534
I20260806 22:49:12.513342 15399 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:49:12.514499 15391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:12.513131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34949 (local address 127.0.0.1:33227)
0806 22:49:12.513179 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:12.513181 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:12.513335 (+ 154us) server_negotiation.cc:407] Connection header received
0806 22:49:12.513450 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:12.513452 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:12.513459 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:12.513518 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:12.514251 (+ 733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:12.514252 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:12.514252 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:49:12.514257 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:12.514276 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:49:12.514281 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:12.514350 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:12.514407 (+ 57us) server_negotiation.cc:299] Negotiation successful
0806 22:49:12.514425 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 22:49:12.523722 15399 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:49:12.523921 15399 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:49:12.531265 15399 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eb89ead419774f0ebac1be475b98d199: Generated new TSK 0
I20260806 22:49:12.531500 15399 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:49:12.535403 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 15297
2026-08-06T22:49:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1651 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056275803658-18615-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:49:12 dist-test-slave-h3ft krb5kdc[15428](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:49:12 dist-test-slave-h3ft krb5kdc[15428](info): set up 2 sockets
Aug 06 22:49:12 dist-test-slave-h3ft krb5kdc[15428](info): commencing operation
krb5kdc: starting...
W20260806 22:49:13.799834 18615 mini_kdc.cc:121] Time spent starting KDC: real 1.234s user 0.001s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Aug 06 22:49:13 dist-test-slave-h3ft krb5kdc[15428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056553, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:49:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:49:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056275803658-18615-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056275803658-18615-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:49:13.946267 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056275803658-18615-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056275803658-18615-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056275803658-18615-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37729
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056275803658-18615-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37729
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:49:14.041267 15445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:14.041584 15445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:14.041625 15445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:14.042809 15445 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:49:14.042940 15445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:14.042969 15445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:14.042999 15445 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:49:14.043048 15445 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:49:14.044693 15445 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056275803658-18615-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056275803658-18615-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37729
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056275803658-18615-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15445
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056275803658-18615-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:49:14.045022 15445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:14.045300 15445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:14.050936 15452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:14.050935 15456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:14.051051 15453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:14.051287 15445 server_base.cc:1034] running on GCE node
I20260806 22:49:14.051640 15445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:14.051983 15445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:14.053159 15445 hybrid_clock.cc:648] HybridClock initialized: now 1786056554053137 us; error 43 us; skew 500 ppm
Aug 06 22:49:14 dist-test-slave-h3ft krb5kdc[15428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056554, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:14.059777 15445 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:49:14.060344 15445 webserver.cc:460] Webserver started at http://127.0.0.1:38579/ using document root <none> and password file <none>
I20260806 22:49:14.060642 15445 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:14.060724 15445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:14.060959 15445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:49:14.062304 15445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056275803658-18615-0/minicluster-data/master-0/data/instance:
uuid: "43a23df7de0c422fa503e2c106a1fc8e"
format_stamp: "Formatted at 2026-08-06 22:49:14 on dist-test-slave-h3ft"
server_key: "7cd1467fd7f2c4220ce80e2bd12c8d7d"
server_key_iv: "eb77fc77668ed55ebf6961e092627d4e"
server_key_version: "encryptionkey@0"
I20260806 22:49:14.062734 15445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056275803658-18615-0/minicluster-data/master-0/wal/instance:
uuid: "43a23df7de0c422fa503e2c106a1fc8e"
format_stamp: "Formatted at 2026-08-06 22:49:14 on dist-test-slave-h3ft"
server_key: "7cd1467fd7f2c4220ce80e2bd12c8d7d"
server_key_iv: "eb77fc77668ed55ebf6961e092627d4e"
server_key_version: "encryptionkey@0"
I20260806 22:49:14.064469 15445 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260806 22:49:14.065459 15464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:14.065681 15445 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:49:14.065805 15445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056275803658-18615-0/minicluster-data/master-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056275803658-18615-0/minicluster-data/master-0/wal
uuid: "43a23df7de0c422fa503e2c106a1fc8e"
format_stamp: "Formatted at 2026-08-06 22:49:14 on dist-test-slave-h3ft"
server_key: "7cd1467fd7f2c4220ce80e2bd12c8d7d"
server_key_iv: "eb77fc77668ed55ebf6961e092627d4e"
server_key_version: "encryptionkey@0"
I20260806 22:49:14.065936 15445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056275803658-18615-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056275803658-18615-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056275803658-18615-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:14.128770 15445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:14.131960 15445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:14.132179 15445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:14.137347 15445 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37729
I20260806 22:49:14.137384 15535 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37729 every 8 connection(s)
I20260806 22:49:14.137815 15445 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
I20260806 22:49:14.139065 15536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:14.142174 15536 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43a23df7de0c422fa503e2c106a1fc8e: Bootstrap starting.
I20260806 22:49:14.143046 15536 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 43a23df7de0c422fa503e2c106a1fc8e: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:14.143435 15536 log.cc:824] T 00000000000000000000000000000000 P 43a23df7de0c422fa503e2c106a1fc8e: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:14.144299 15536 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43a23df7de0c422fa503e2c106a1fc8e: No bootstrap required, opened a new log
I20260806 22:49:14.144755 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 15445
I20260806 22:49:14.144948 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786056275803658-18615-0/minicluster-data/master-0/wal/instance
I20260806 22:49:14.145187 18615 external_mini_cluster.cc:1428] Setting key 56fb6c55fdd8ee0826c22401fb06a757
I20260806 22:49:14.146993 15536 raft_consensus.cc:348] T 00000000000000000000000000000000 P 43a23df7de0c422fa503e2c106a1fc8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43a23df7de0c422fa503e2c106a1fc8e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37729 } }
I20260806 22:49:14.147244 15536 raft_consensus.cc:374] T 00000000000000000000000000000000 P 43a23df7de0c422fa503e2c106a1fc8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:14.147292 15536 raft_consensus.cc:729] T 00000000000000000000000000000000 P 43a23df7de0c422fa503e2c106a1fc8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43a23df7de0c422fa503e2c106a1fc8e, State: Initialized, Role: FOLLOWER
I20260806 22:49:14.147511 15536 consensus_queue.cc:260] T 00000000000000000000000000000000 P 43a23df7de0c422fa503e2c106a1fc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43a23df7de0c422fa503e2c106a1fc8e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37729 } }
I20260806 22:49:14.147607 15536 raft_consensus.cc:388] T 00000000000000000000000000000000 P 43a23df7de0c422fa503e2c106a1fc8e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:49:14.147634 15536 raft_consensus.cc:482] T 00000000000000000000000000000000 P 43a23df7de0c422fa503e2c106a1fc8e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:49:14.147708 15536 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 43a23df7de0c422fa503e2c106a1fc8e [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:14.149137 15536 raft_consensus.cc:504] T 00000000000000000000000000000000 P 43a23df7de0c422fa503e2c106a1fc8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43a23df7de0c422fa503e2c106a1fc8e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37729 } }
I20260806 22:49:14.149267 15536 leader_election.cc:302] T 00000000000000000000000000000000 P 43a23df7de0c422fa503e2c106a1fc8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 43a23df7de0c422fa503e2c106a1fc8e; no voters:
I20260806 22:49:14.149451 15536 leader_election.cc:288] T 00000000000000000000000000000000 P 43a23df7de0c422fa503e2c106a1fc8e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:49:14.149540 15545 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 43a23df7de0c422fa503e2c106a1fc8e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:49:14.149741 15536 sys_catalog.cc:564] T 00000000000000000000000000000000 P 43a23df7de0c422fa503e2c106a1fc8e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:49:14.149765 15545 raft_consensus.cc:686] T 00000000000000000000000000000000 P 43a23df7de0c422fa503e2c106a1fc8e [term 1 LEADER]: Becoming Leader. State: Replica: 43a23df7de0c422fa503e2c106a1fc8e, State: Running, Role: LEADER
I20260806 22:49:14.150530 15545 consensus_queue.cc:237] T 00000000000000000000000000000000 P 43a23df7de0c422fa503e2c106a1fc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43a23df7de0c422fa503e2c106a1fc8e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37729 } }
I20260806 22:49:14.151410 15546 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43a23df7de0c422fa503e2c106a1fc8e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "43a23df7de0c422fa503e2c106a1fc8e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43a23df7de0c422fa503e2c106a1fc8e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37729 } } }
I20260806 22:49:14.151590 15547 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43a23df7de0c422fa503e2c106a1fc8e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 43a23df7de0c422fa503e2c106a1fc8e. Latest consensus state: current_term: 1 leader_uuid: "43a23df7de0c422fa503e2c106a1fc8e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43a23df7de0c422fa503e2c106a1fc8e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37729 } } }
I20260806 22:49:14.152436 15546 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43a23df7de0c422fa503e2c106a1fc8e [sys.catalog]: This master's current role is: LEADER
I20260806 22:49:14.152451 15547 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43a23df7de0c422fa503e2c106a1fc8e [sys.catalog]: This master's current role is: LEADER
W20260806 22:49:14.152063 15561 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 43a23df7de0c422fa503e2c106a1fc8e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:49:14.152710 15561 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:49:14.153810 15564 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:49:14.154057 15564 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 22:49:14 dist-test-slave-h3ft krb5kdc[15428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056553, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 22:49:14.155997 15564 catalog_manager.cc:1347] Generated new cluster ID: 005769da3b3e4659b21126d9c1b4a5b1
I20260806 22:49:14.156059 15564 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:49:14.159075 15544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:14.145881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35837 (local address 127.0.0.1:37729)
0806 22:49:14.146147 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:14.146160 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:14.146205 (+ 45us) server_negotiation.cc:407] Connection header received
0806 22:49:14.146944 (+ 739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:14.146954 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:14.147070 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:14.147249 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:14.148227 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:14.149777 (+ 1550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:14.150836 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:14.151538 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:14.151668 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:14.155106 (+ 3438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:14.155111 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:14.155111 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:14.155125 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:14.157166 (+ 2041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:14.157569 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:14.157573 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:14.157573 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:14.157707 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:14.157948 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:14.157949 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:14.157949 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:14.158215 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:14.158419 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:14.158491 (+ 72us) server_negotiation.cc:299] Negotiation successful
0806 22:49:14.158647 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":127,"threads_started":1}
I20260806 22:49:14.160250 18615 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:49:14.168845 15544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:14.161966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43261 (local address 127.0.0.1:37729)
0806 22:49:14.162024 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:14.162027 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:14.162280 (+ 253us) server_negotiation.cc:407] Connection header received
0806 22:49:14.162358 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:14.162359 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:14.162371 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:14.162417 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:14.162982 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:14.163556 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:14.164296 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:14.164545 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:14.164628 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:14.165978 (+ 1350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:14.165980 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:14.165980 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:14.165988 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:14.167548 (+ 1560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:14.167979 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:14.167981 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:14.167982 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:14.168047 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:14.168263 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:14.168264 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:14.168265 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:14.168396 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:14.168528 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:14.168648 (+ 120us) server_negotiation.cc:299] Negotiation successful
0806 22:49:14.168733 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 22:49:14.178704 15564 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:49:14.179592 15564 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:49:14.186591 15564 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 43a23df7de0c422fa503e2c106a1fc8e: Generated new TSK 0
I20260806 22:49:14.186902 15564 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:49:14.198276 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 15445
2026-08-06T22:49:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1662 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056275803658-18615-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:49:14 dist-test-slave-h3ft krb5kdc[15575](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:49:14 dist-test-slave-h3ft krb5kdc[15575](info): set up 2 sockets
Aug 06 22:49:14 dist-test-slave-h3ft krb5kdc[15575](info): commencing operation
krb5kdc: starting...
W20260806 22:49:15.534958 18615 mini_kdc.cc:121] Time spent starting KDC: real 1.306s user 0.001s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Aug 06 22:49:15 dist-test-slave-h3ft krb5kdc[15575](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056555, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:49:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:49:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056275803658-18615-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056275803658-18615-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:49:15.682436 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056275803658-18615-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056275803658-18615-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056275803658-18615-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35289
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056275803658-18615-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35289
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:49:15.782168 15593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:15.782501 15593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:15.782547 15593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:15.783986 15593 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:49:15.784078 15593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:15.784101 15593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:15.784122 15593 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:49:15.784149 15593 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:49:15.785871 15593 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056275803658-18615-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056275803658-18615-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35289
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056275803658-18615-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35289
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15593
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056275803658-18615-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:49:15.786240 15593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:15.786540 15593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:15.792583 15602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:15.792635 15601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:15.792825 15593 server_base.cc:1034] running on GCE node
W20260806 22:49:15.792884 15604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:15.793224 15593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:15.793607 15593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:15.794780 15593 hybrid_clock.cc:648] HybridClock initialized: now 1786056555794742 us; error 50 us; skew 500 ppm
Aug 06 22:49:15 dist-test-slave-h3ft krb5kdc[15575](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056555, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:15.798050 15593 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:49:15.798642 15593 webserver.cc:460] Webserver started at http://127.0.0.1:36341/ using document root <none> and password file <none>
I20260806 22:49:15.798933 15593 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:15.799044 15593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:15.799288 15593 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:49:15.800592 15593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056275803658-18615-0/minicluster-data/master-0/data/instance:
uuid: "56a8f8ab03864b34b20cafba7f4455cd"
format_stamp: "Formatted at 2026-08-06 22:49:15 on dist-test-slave-h3ft"
server_key: "9a48e785bb002fb174dd9a82073aac57"
server_key_iv: "bdc4f3f9edfe8aaff564e875219032f2"
server_key_version: "encryptionkey@0"
I20260806 22:49:15.801026 15593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056275803658-18615-0/minicluster-data/master-0/wal/instance:
uuid: "56a8f8ab03864b34b20cafba7f4455cd"
format_stamp: "Formatted at 2026-08-06 22:49:15 on dist-test-slave-h3ft"
server_key: "9a48e785bb002fb174dd9a82073aac57"
server_key_iv: "bdc4f3f9edfe8aaff564e875219032f2"
server_key_version: "encryptionkey@0"
I20260806 22:49:15.802640 15593 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:49:15.803737 15616 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:15.804008 15593 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:49:15.804140 15593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056275803658-18615-0/minicluster-data/master-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056275803658-18615-0/minicluster-data/master-0/wal
uuid: "56a8f8ab03864b34b20cafba7f4455cd"
format_stamp: "Formatted at 2026-08-06 22:49:15 on dist-test-slave-h3ft"
server_key: "9a48e785bb002fb174dd9a82073aac57"
server_key_iv: "bdc4f3f9edfe8aaff564e875219032f2"
server_key_version: "encryptionkey@0"
I20260806 22:49:15.804230 15593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056275803658-18615-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056275803658-18615-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056275803658-18615-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:15.829974 15593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:15.833061 15593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:15.833268 15593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:15.838814 15593 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35289
I20260806 22:49:15.838848 15687 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35289 every 8 connection(s)
I20260806 22:49:15.839354 15593 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
I20260806 22:49:15.840498 15688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:15.840595 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 15593
I20260806 22:49:15.840799 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786056275803658-18615-0/minicluster-data/master-0/wal/instance
I20260806 22:49:15.841037 18615 external_mini_cluster.cc:1428] Setting key b062cdaf912a059b5ef7b0a82d10867d
I20260806 22:49:15.844508 15688 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56a8f8ab03864b34b20cafba7f4455cd: Bootstrap starting.
I20260806 22:49:15.845381 15688 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 56a8f8ab03864b34b20cafba7f4455cd: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:15.845760 15688 log.cc:824] T 00000000000000000000000000000000 P 56a8f8ab03864b34b20cafba7f4455cd: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:15.846685 15688 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56a8f8ab03864b34b20cafba7f4455cd: No bootstrap required, opened a new log
I20260806 22:49:15.847707 15693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:15.841717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47125 (local address 127.0.0.1:35289)
0806 22:49:15.841928 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:15.841939 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:15.842004 (+ 65us) server_negotiation.cc:407] Connection header received
0806 22:49:15.842584 (+ 580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:15.842592 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:15.842652 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:15.842797 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:15.843099 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:15.844090 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:15.845249 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:15.845784 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:15.845933 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:15.846810 (+ 877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:15.846822 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:15.846822 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:49:15.847088 (+ 266us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:15.847142 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:49:15.847156 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:15.847212 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:15.847323 (+ 111us) server_negotiation.cc:299] Negotiation successful
0806 22:49:15.847437 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":108,"threads_started":1}
I20260806 22:49:15.848945 18615 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:49:15.849602 15688 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56a8f8ab03864b34b20cafba7f4455cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56a8f8ab03864b34b20cafba7f4455cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35289 } }
I20260806 22:49:15.849818 15688 raft_consensus.cc:374] T 00000000000000000000000000000000 P 56a8f8ab03864b34b20cafba7f4455cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:15.849852 15688 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56a8f8ab03864b34b20cafba7f4455cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56a8f8ab03864b34b20cafba7f4455cd, State: Initialized, Role: FOLLOWER
I20260806 22:49:15.849998 15688 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56a8f8ab03864b34b20cafba7f4455cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56a8f8ab03864b34b20cafba7f4455cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35289 } }
I20260806 22:49:15.850096 15688 raft_consensus.cc:388] T 00000000000000000000000000000000 P 56a8f8ab03864b34b20cafba7f4455cd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:49:15.850131 15688 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56a8f8ab03864b34b20cafba7f4455cd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:49:15.850181 15688 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56a8f8ab03864b34b20cafba7f4455cd [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:15.851118 15688 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56a8f8ab03864b34b20cafba7f4455cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56a8f8ab03864b34b20cafba7f4455cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35289 } }
I20260806 22:49:15.851258 15688 leader_election.cc:302] T 00000000000000000000000000000000 P 56a8f8ab03864b34b20cafba7f4455cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 56a8f8ab03864b34b20cafba7f4455cd; no voters:
I20260806 22:49:15.851451 15688 leader_election.cc:288] T 00000000000000000000000000000000 P 56a8f8ab03864b34b20cafba7f4455cd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:49:15.851485 15703 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 56a8f8ab03864b34b20cafba7f4455cd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:49:15.851681 15703 raft_consensus.cc:686] T 00000000000000000000000000000000 P 56a8f8ab03864b34b20cafba7f4455cd [term 1 LEADER]: Becoming Leader. State: Replica: 56a8f8ab03864b34b20cafba7f4455cd, State: Running, Role: LEADER
I20260806 22:49:15.851846 15688 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56a8f8ab03864b34b20cafba7f4455cd [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:49:15.851832 15703 consensus_queue.cc:237] T 00000000000000000000000000000000 P 56a8f8ab03864b34b20cafba7f4455cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56a8f8ab03864b34b20cafba7f4455cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35289 } }
I20260806 22:49:15.854115 15706 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56a8f8ab03864b34b20cafba7f4455cd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "56a8f8ab03864b34b20cafba7f4455cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56a8f8ab03864b34b20cafba7f4455cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35289 } } }
I20260806 22:49:15.854239 15706 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56a8f8ab03864b34b20cafba7f4455cd [sys.catalog]: This master's current role is: LEADER
I20260806 22:49:15.854115 15705 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56a8f8ab03864b34b20cafba7f4455cd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 56a8f8ab03864b34b20cafba7f4455cd. Latest consensus state: current_term: 1 leader_uuid: "56a8f8ab03864b34b20cafba7f4455cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56a8f8ab03864b34b20cafba7f4455cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35289 } } }
I20260806 22:49:15.854477 15693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:15.850942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48707 (local address 127.0.0.1:35289)
0806 22:49:15.851034 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:15.851036 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:15.851196 (+ 160us) server_negotiation.cc:407] Connection header received
0806 22:49:15.851271 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:15.851272 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:15.851286 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:15.851354 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:15.851522 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:15.852196 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:15.852982 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:15.853276 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:15.853356 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:15.854176 (+ 820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:15.854177 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:15.854178 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:49:15.854183 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:15.854202 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:49:15.854208 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:15.854256 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:15.854351 (+ 95us) server_negotiation.cc:299] Negotiation successful
0806 22:49:15.854382 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 22:49:15.854496 15705 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56a8f8ab03864b34b20cafba7f4455cd [sys.catalog]: This master's current role is: LEADER
I20260806 22:49:15.854868 15722 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:49:15.855492 15722 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:49:15.857182 15722 catalog_manager.cc:1347] Generated new cluster ID: 22d37e36b11a43139732500ddafb1ce2
I20260806 22:49:15.857239 15722 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:49:15.872417 15722 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:49:15.873291 15722 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:49:15.877909 15722 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 56a8f8ab03864b34b20cafba7f4455cd: Generated new TSK 0
I20260806 22:49:15.878160 15722 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:49:15.882252 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 15593
2026-08-06T22:49:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1688 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056275803658-18615-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:49:15 dist-test-slave-h3ft krb5kdc[15731](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:49:15 dist-test-slave-h3ft krb5kdc[15731](info): set up 2 sockets
Aug 06 22:49:15 dist-test-slave-h3ft krb5kdc[15731](info): commencing operation
krb5kdc: starting...
W20260806 22:49:17.153507 18615 mini_kdc.cc:121] Time spent starting KDC: real 1.237s user 0.001s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Aug 06 22:49:17 dist-test-slave-h3ft krb5kdc[15731](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056557, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:49:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:49:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056275803658-18615-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056275803658-18615-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:49:17.296386 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056275803658-18615-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056275803658-18615-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056275803658-18615-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39023
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056275803658-18615-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39023
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:49:17.393474 15747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:17.393788 15747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:17.393831 15747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:17.395126 15747 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:49:17.395202 15747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:17.395226 15747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:17.395246 15747 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:49:17.395279 15747 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:49:17.396910 15747 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056275803658-18615-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056275803658-18615-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39023
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056275803658-18615-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39023
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15747
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056275803658-18615-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:49:17.397248 15747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:17.397537 15747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:17.403301 15752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:17.403293 15756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:17.403297 15753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:17.403764 15747 server_base.cc:1034] running on GCE node
I20260806 22:49:17.404114 15747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:17.404471 15747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:17.405644 15747 hybrid_clock.cc:648] HybridClock initialized: now 1786056557405613 us; error 48 us; skew 500 ppm
Aug 06 22:49:17 dist-test-slave-h3ft krb5kdc[15731](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056557, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:17.408880 15747 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:49:17.409472 15747 webserver.cc:460] Webserver started at http://127.0.0.1:34669/ using document root <none> and password file <none>
I20260806 22:49:17.409736 15747 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:17.409782 15747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:17.409916 15747 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:49:17.411229 15747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056275803658-18615-0/minicluster-data/master-0/data/instance:
uuid: "31472eec8e7a4649a9b726d279f9f5eb"
format_stamp: "Formatted at 2026-08-06 22:49:17 on dist-test-slave-h3ft"
server_key: "e3e672cf26df4ef0b319c60e30aa1aed"
server_key_iv: "c7b8ce19ec4a35e12addd3a69197717b"
server_key_version: "encryptionkey@0"
I20260806 22:49:17.411586 15747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056275803658-18615-0/minicluster-data/master-0/wal/instance:
uuid: "31472eec8e7a4649a9b726d279f9f5eb"
format_stamp: "Formatted at 2026-08-06 22:49:17 on dist-test-slave-h3ft"
server_key: "e3e672cf26df4ef0b319c60e30aa1aed"
server_key_iv: "c7b8ce19ec4a35e12addd3a69197717b"
server_key_version: "encryptionkey@0"
I20260806 22:49:17.413257 15747 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 22:49:17.414196 15767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:17.414461 15747 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:49:17.414592 15747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056275803658-18615-0/minicluster-data/master-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056275803658-18615-0/minicluster-data/master-0/wal
uuid: "31472eec8e7a4649a9b726d279f9f5eb"
format_stamp: "Formatted at 2026-08-06 22:49:17 on dist-test-slave-h3ft"
server_key: "e3e672cf26df4ef0b319c60e30aa1aed"
server_key_iv: "c7b8ce19ec4a35e12addd3a69197717b"
server_key_version: "encryptionkey@0"
I20260806 22:49:17.414700 15747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056275803658-18615-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056275803658-18615-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056275803658-18615-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:17.418486 15747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:17.421147 15747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:17.421285 15747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:17.426573 15747 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39023
I20260806 22:49:17.426615 15839 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39023 every 8 connection(s)
I20260806 22:49:17.427134 15747 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
I20260806 22:49:17.428537 15841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:17.431509 15841 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31472eec8e7a4649a9b726d279f9f5eb: Bootstrap starting.
I20260806 22:49:17.432272 15841 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 31472eec8e7a4649a9b726d279f9f5eb: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:17.432642 15841 log.cc:824] T 00000000000000000000000000000000 P 31472eec8e7a4649a9b726d279f9f5eb: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:17.433472 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 15747
I20260806 22:49:17.433622 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786056275803658-18615-0/minicluster-data/master-0/wal/instance
I20260806 22:49:17.433720 15841 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31472eec8e7a4649a9b726d279f9f5eb: No bootstrap required, opened a new log
I20260806 22:49:17.433856 18615 external_mini_cluster.cc:1428] Setting key c9cc58e50cf564da9933ec241a8030c7
I20260806 22:49:17.436128 15841 raft_consensus.cc:348] T 00000000000000000000000000000000 P 31472eec8e7a4649a9b726d279f9f5eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31472eec8e7a4649a9b726d279f9f5eb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39023 } }
I20260806 22:49:17.436331 15841 raft_consensus.cc:374] T 00000000000000000000000000000000 P 31472eec8e7a4649a9b726d279f9f5eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:17.436383 15841 raft_consensus.cc:729] T 00000000000000000000000000000000 P 31472eec8e7a4649a9b726d279f9f5eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31472eec8e7a4649a9b726d279f9f5eb, State: Initialized, Role: FOLLOWER
I20260806 22:49:17.436594 15841 consensus_queue.cc:260] T 00000000000000000000000000000000 P 31472eec8e7a4649a9b726d279f9f5eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31472eec8e7a4649a9b726d279f9f5eb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39023 } }
I20260806 22:49:17.436750 15841 raft_consensus.cc:388] T 00000000000000000000000000000000 P 31472eec8e7a4649a9b726d279f9f5eb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:49:17.436805 15841 raft_consensus.cc:482] T 00000000000000000000000000000000 P 31472eec8e7a4649a9b726d279f9f5eb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:49:17.436864 15841 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 31472eec8e7a4649a9b726d279f9f5eb [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:17.437911 15846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:17.434644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58135 (local address 127.0.0.1:39023)
0806 22:49:17.434897 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:17.434906 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:17.434938 (+ 32us) server_negotiation.cc:407] Connection header received
0806 22:49:17.436118 (+ 1180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:17.436129 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:17.436169 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:17.436355 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:17.437237 (+ 882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:17.437242 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:17.437242 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:49:17.437247 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:17.437295 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:49:17.437309 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:17.437391 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:17.437519 (+ 128us) server_negotiation.cc:299] Negotiation successful
0806 22:49:17.437598 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":115,"threads_started":1}
I20260806 22:49:17.438225 15841 raft_consensus.cc:504] T 00000000000000000000000000000000 P 31472eec8e7a4649a9b726d279f9f5eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31472eec8e7a4649a9b726d279f9f5eb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39023 } }
I20260806 22:49:17.438364 15841 leader_election.cc:302] T 00000000000000000000000000000000 P 31472eec8e7a4649a9b726d279f9f5eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 31472eec8e7a4649a9b726d279f9f5eb; no voters:
I20260806 22:49:17.438597 15841 leader_election.cc:288] T 00000000000000000000000000000000 P 31472eec8e7a4649a9b726d279f9f5eb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:49:17.438808 15847 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 31472eec8e7a4649a9b726d279f9f5eb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:49:17.438925 15841 sys_catalog.cc:564] T 00000000000000000000000000000000 P 31472eec8e7a4649a9b726d279f9f5eb [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:49:17.439052 18615 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:49:17.439069 15847 raft_consensus.cc:686] T 00000000000000000000000000000000 P 31472eec8e7a4649a9b726d279f9f5eb [term 1 LEADER]: Becoming Leader. State: Replica: 31472eec8e7a4649a9b726d279f9f5eb, State: Running, Role: LEADER
I20260806 22:49:17.439165 15847 consensus_queue.cc:237] T 00000000000000000000000000000000 P 31472eec8e7a4649a9b726d279f9f5eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31472eec8e7a4649a9b726d279f9f5eb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39023 } }
I20260806 22:49:17.439839 15850 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31472eec8e7a4649a9b726d279f9f5eb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 31472eec8e7a4649a9b726d279f9f5eb. Latest consensus state: current_term: 1 leader_uuid: "31472eec8e7a4649a9b726d279f9f5eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31472eec8e7a4649a9b726d279f9f5eb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39023 } } }
I20260806 22:49:17.439952 15850 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31472eec8e7a4649a9b726d279f9f5eb [sys.catalog]: This master's current role is: LEADER
I20260806 22:49:17.440116 15848 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31472eec8e7a4649a9b726d279f9f5eb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "31472eec8e7a4649a9b726d279f9f5eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31472eec8e7a4649a9b726d279f9f5eb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39023 } } }
I20260806 22:49:17.440172 15848 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31472eec8e7a4649a9b726d279f9f5eb [sys.catalog]: This master's current role is: LEADER
W20260806 22:49:17.441291 15874 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 31472eec8e7a4649a9b726d279f9f5eb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:49:17.441365 15874 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:49:17.441417 15858 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:49:17.441606 15858 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:49:17.442551 15846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:17.440957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48243 (local address 127.0.0.1:39023)
0806 22:49:17.441190 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:17.441192 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:17.441261 (+ 69us) server_negotiation.cc:407] Connection header received
0806 22:49:17.441344 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:17.441346 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:17.441352 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:17.441406 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:17.442240 (+ 834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:17.442241 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:17.442242 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:49:17.442247 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:17.442267 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:49:17.442272 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:17.442338 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:17.442463 (+ 125us) server_negotiation.cc:299] Negotiation successful
0806 22:49:17.442480 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209}
I20260806 22:49:17.443498 15858 catalog_manager.cc:1347] Generated new cluster ID: 1329371b1e4641f4b73722c1665de886
I20260806 22:49:17.443564 15858 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:49:17.483415 15858 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:49:17.483688 15858 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:49:17.491974 15858 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 31472eec8e7a4649a9b726d279f9f5eb: Generated new TSK 0
I20260806 22:49:17.492271 15858 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:49:17.505517 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 15747
2026-08-06T22:49:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1618 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056275803658-18615-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:49:17 dist-test-slave-h3ft krb5kdc[15883](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:49:17 dist-test-slave-h3ft krb5kdc[15883](info): set up 2 sockets
Aug 06 22:49:17 dist-test-slave-h3ft krb5kdc[15883](info): commencing operation
krb5kdc: starting...
W20260806 22:49:18.750412 18615 mini_kdc.cc:121] Time spent starting KDC: real 1.215s user 0.006s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Aug 06 22:49:18 dist-test-slave-h3ft krb5kdc[15883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056558, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:49:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:49:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056275803658-18615-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056275803658-18615-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:49:18.902981 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056275803658-18615-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056275803658-18615-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056275803658-18615-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41327
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056275803658-18615-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41327
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:49:18.999622 15897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:19.000066 15897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:19.000146 15897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:19.002008 15897 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:49:19.002058 15897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:19.002074 15897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:19.002086 15897 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:49:19.002099 15897 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:49:19.003657 15897 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056275803658-18615-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056275803658-18615-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41327
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056275803658-18615-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41327
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15897
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056275803658-18615-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:49:19.003913 15897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:19.004134 15897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:19.010062 15909 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:19.010078 15904 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:19.010298 15897 server_base.cc:1034] running on GCE node
W20260806 22:49:19.010140 15905 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:19.010740 15897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:19.011150 15897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:19.012336 15897 hybrid_clock.cc:648] HybridClock initialized: now 1786056559012325 us; error 45 us; skew 500 ppm
Aug 06 22:49:19 dist-test-slave-h3ft krb5kdc[15883](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056559, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:19.015694 15897 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:49:19.016285 15897 webserver.cc:460] Webserver started at http://127.0.0.1:41163/ using document root <none> and password file <none>
I20260806 22:49:19.016561 15897 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:19.016635 15897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:19.016808 15897 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:49:19.018251 15897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056275803658-18615-0/minicluster-data/master-0/data/instance:
uuid: "8a04c97d0123414494c152926ce99870"
format_stamp: "Formatted at 2026-08-06 22:49:19 on dist-test-slave-h3ft"
server_key: "fe55659b3c5bd7a90c2cb670bbea4f45"
server_key_iv: "1051ec7a98f4b52d021f18e1d64a7f98"
server_key_version: "encryptionkey@0"
I20260806 22:49:19.018748 15897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056275803658-18615-0/minicluster-data/master-0/wal/instance:
uuid: "8a04c97d0123414494c152926ce99870"
format_stamp: "Formatted at 2026-08-06 22:49:19 on dist-test-slave-h3ft"
server_key: "fe55659b3c5bd7a90c2cb670bbea4f45"
server_key_iv: "1051ec7a98f4b52d021f18e1d64a7f98"
server_key_version: "encryptionkey@0"
I20260806 22:49:19.020488 15897 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260806 22:49:19.021538 15923 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:19.021816 15897 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:49:19.021978 15897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056275803658-18615-0/minicluster-data/master-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056275803658-18615-0/minicluster-data/master-0/wal
uuid: "8a04c97d0123414494c152926ce99870"
format_stamp: "Formatted at 2026-08-06 22:49:19 on dist-test-slave-h3ft"
server_key: "fe55659b3c5bd7a90c2cb670bbea4f45"
server_key_iv: "1051ec7a98f4b52d021f18e1d64a7f98"
server_key_version: "encryptionkey@0"
I20260806 22:49:19.022082 15897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056275803658-18615-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056275803658-18615-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056275803658-18615-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:19.039942 15897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:19.042836 15897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:19.043067 15897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:19.048544 15897 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41327
I20260806 22:49:19.048593 15994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41327 every 8 connection(s)
I20260806 22:49:19.049072 15897 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
I20260806 22:49:19.050242 15995 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:19.051271 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 15897
I20260806 22:49:19.051451 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786056275803658-18615-0/minicluster-data/master-0/wal/instance
I20260806 22:49:19.051679 18615 external_mini_cluster.cc:1428] Setting key d47f4fb11671fd8326069c5a91c0656f
I20260806 22:49:19.054267 15995 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a04c97d0123414494c152926ce99870: Bootstrap starting.
I20260806 22:49:19.055209 15995 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8a04c97d0123414494c152926ce99870: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:19.055588 15995 log.cc:824] T 00000000000000000000000000000000 P 8a04c97d0123414494c152926ce99870: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:19.056465 15995 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a04c97d0123414494c152926ce99870: No bootstrap required, opened a new log
I20260806 22:49:19.058931 15995 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a04c97d0123414494c152926ce99870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a04c97d0123414494c152926ce99870" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41327 } }
I20260806 22:49:19.059172 15995 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8a04c97d0123414494c152926ce99870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:19.059206 15995 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a04c97d0123414494c152926ce99870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a04c97d0123414494c152926ce99870, State: Initialized, Role: FOLLOWER
I20260806 22:49:19.059365 15995 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a04c97d0123414494c152926ce99870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a04c97d0123414494c152926ce99870" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41327 } }
I20260806 22:49:19.059451 15995 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8a04c97d0123414494c152926ce99870 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:49:19.059496 15995 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a04c97d0123414494c152926ce99870 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:49:19.059552 15995 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a04c97d0123414494c152926ce99870 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:19.060227 15995 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a04c97d0123414494c152926ce99870 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a04c97d0123414494c152926ce99870" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41327 } }
I20260806 22:49:19.060355 15995 leader_election.cc:302] T 00000000000000000000000000000000 P 8a04c97d0123414494c152926ce99870 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8a04c97d0123414494c152926ce99870; no voters:
I20260806 22:49:19.060560 15995 leader_election.cc:288] T 00000000000000000000000000000000 P 8a04c97d0123414494c152926ce99870 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:49:19.060683 16002 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a04c97d0123414494c152926ce99870 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:49:19.060889 16002 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8a04c97d0123414494c152926ce99870 [term 1 LEADER]: Becoming Leader. State: Replica: 8a04c97d0123414494c152926ce99870, State: Running, Role: LEADER
I20260806 22:49:19.060927 15995 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a04c97d0123414494c152926ce99870 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:49:19.061017 16002 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a04c97d0123414494c152926ce99870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a04c97d0123414494c152926ce99870" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41327 } }
I20260806 22:49:19.061486 16004 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a04c97d0123414494c152926ce99870 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a04c97d0123414494c152926ce99870. Latest consensus state: current_term: 1 leader_uuid: "8a04c97d0123414494c152926ce99870" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a04c97d0123414494c152926ce99870" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41327 } } }
I20260806 22:49:19.061625 16004 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a04c97d0123414494c152926ce99870 [sys.catalog]: This master's current role is: LEADER
I20260806 22:49:19.061902 16014 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:49:19.061479 16003 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a04c97d0123414494c152926ce99870 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8a04c97d0123414494c152926ce99870" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a04c97d0123414494c152926ce99870" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41327 } } }
I20260806 22:49:19.062283 16003 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a04c97d0123414494c152926ce99870 [sys.catalog]: This master's current role is: LEADER
I20260806 22:49:19.062553 16014 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:49:19.064533 16014 catalog_manager.cc:1347] Generated new cluster ID: e809120bae54487b8384277a9bdd18ec
I20260806 22:49:19.064599 16014 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:49:19 dist-test-slave-h3ft krb5kdc[15883](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056558, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 22:49:19.072532 15999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:19.052442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.166:38203 (local address 127.0.0.1:41327)
0806 22:49:19.052697 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:19.052710 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:19.052751 (+ 41us) server_negotiation.cc:407] Connection header received
0806 22:49:19.053385 (+ 634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:19.053396 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:19.053479 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:19.053654 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:19.054387 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:19.055916 (+ 1529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:19.056803 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:19.057458 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:19.057582 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:19.067688 (+ 10106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:19.067696 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:19.067697 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:19.067715 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:19.070243 (+ 2528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:19.070851 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:19.070861 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:19.070862 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:19.071037 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:19.071316 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:19.071317 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:19.071317 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:19.071762 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:19.071971 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:19.072109 (+ 138us) server_negotiation.cc:299] Negotiation successful
0806 22:49:19.072238 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":129,"threads_started":1}
I20260806 22:49:19.074120 18615 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:49:19.074206 16014 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:49:19.074952 16014 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:49:19.083335 15999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:19.076207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.166:37035 (local address 127.0.0.1:41327)
0806 22:49:19.076251 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:19.076253 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:19.076262 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:49:19.076327 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:19.076328 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:19.076340 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:19.076398 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:19.077021 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:19.077590 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:19.078376 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:19.078638 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:19.078725 (+ 87us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:19.080231 (+ 1506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:19.080233 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:19.080233 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:19.080242 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:19.081904 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:19.082346 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:19.082348 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:19.082348 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:19.082420 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:19.082671 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:19.082672 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:19.082673 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:19.082803 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:19.082955 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:19.083126 (+ 171us) server_negotiation.cc:299] Negotiation successful
0806 22:49:19.083200 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 22:49:19.086527 16014 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8a04c97d0123414494c152926ce99870: Generated new TSK 0
I20260806 22:49:19.086824 16014 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:49:19.092298 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 15897
W20260806 22:49:19.109018 15880 connection.cc:535] client connection to 127.0.0.1:41327 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41327): Connection reset by peer (error 104)
2026-08-06T22:49:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1589 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056275803658-18615-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:49:19 dist-test-slave-h3ft krb5kdc[16040](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:49:19 dist-test-slave-h3ft krb5kdc[16040](info): set up 2 sockets
Aug 06 22:49:19 dist-test-slave-h3ft krb5kdc[16040](info): commencing operation
krb5kdc: starting...
W20260806 22:49:20.402791 18615 mini_kdc.cc:121] Time spent starting KDC: real 1.278s user 0.001s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Aug 06 22:49:20 dist-test-slave-h3ft krb5kdc[16040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056560, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:49:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:49:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056275803658-18615-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056275803658-18615-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:49:20.546012 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056275803658-18615-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056275803658-18615-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056275803658-18615-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34011
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056275803658-18615-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34011
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:49:20.641360 16056 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:20.641613 16056 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:20.641638 16056 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:20.642797 16056 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:49:20.642840 16056 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:20.642855 16056 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:20.642868 16056 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:49:20.642887 16056 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:49:20.644579 16056 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056275803658-18615-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056275803658-18615-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34011
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056275803658-18615-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34011
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16056
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056275803658-18615-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:49:20.644837 16056 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:20.645066 16056 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:20.650799 16063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:20.650890 16065 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:20.650791 16062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:20.651465 16056 server_base.cc:1034] running on GCE node
I20260806 22:49:20.651782 16056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:20.652102 16056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:20.653347 16056 hybrid_clock.cc:648] HybridClock initialized: now 1786056560653268 us; error 38 us; skew 500 ppm
Aug 06 22:49:20 dist-test-slave-h3ft krb5kdc[16040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056560, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:20.656528 16056 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:49:20.657138 16056 webserver.cc:460] Webserver started at http://127.0.0.1:37469/ using document root <none> and password file <none>
I20260806 22:49:20.657438 16056 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:20.657512 16056 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:20.657688 16056 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:49:20.658918 16056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056275803658-18615-0/minicluster-data/master-0/data/instance:
uuid: "566dbd16450d44bda4b7d6b6e9817ec6"
format_stamp: "Formatted at 2026-08-06 22:49:20 on dist-test-slave-h3ft"
server_key: "ea6ce2d7d16871446b1e514ba1de66da"
server_key_iv: "5b5bb907cf1e234a810c33b2c8392b05"
server_key_version: "encryptionkey@0"
I20260806 22:49:20.659435 16056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056275803658-18615-0/minicluster-data/master-0/wal/instance:
uuid: "566dbd16450d44bda4b7d6b6e9817ec6"
format_stamp: "Formatted at 2026-08-06 22:49:20 on dist-test-slave-h3ft"
server_key: "ea6ce2d7d16871446b1e514ba1de66da"
server_key_iv: "5b5bb907cf1e234a810c33b2c8392b05"
server_key_version: "encryptionkey@0"
I20260806 22:49:20.660992 16056 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:49:20.661880 16079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:20.662082 16056 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:49:20.662207 16056 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056275803658-18615-0/minicluster-data/master-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056275803658-18615-0/minicluster-data/master-0/wal
uuid: "566dbd16450d44bda4b7d6b6e9817ec6"
format_stamp: "Formatted at 2026-08-06 22:49:20 on dist-test-slave-h3ft"
server_key: "ea6ce2d7d16871446b1e514ba1de66da"
server_key_iv: "5b5bb907cf1e234a810c33b2c8392b05"
server_key_version: "encryptionkey@0"
I20260806 22:49:20.662289 16056 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056275803658-18615-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056275803658-18615-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056275803658-18615-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:20.676466 16056 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:20.678905 16056 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:20.679163 16056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:20.684409 16056 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34011
I20260806 22:49:20.684445 16154 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34011 every 8 connection(s)
I20260806 22:49:20.684883 16056 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
I20260806 22:49:20.685993 16155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:20.688963 16155 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 566dbd16450d44bda4b7d6b6e9817ec6: Bootstrap starting.
I20260806 22:49:20.689677 16155 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 566dbd16450d44bda4b7d6b6e9817ec6: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:20.690043 16155 log.cc:824] T 00000000000000000000000000000000 P 566dbd16450d44bda4b7d6b6e9817ec6: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:20.690863 16155 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 566dbd16450d44bda4b7d6b6e9817ec6: No bootstrap required, opened a new log
I20260806 22:49:20.692467 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 16056
I20260806 22:49:20.692588 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786056275803658-18615-0/minicluster-data/master-0/wal/instance
I20260806 22:49:20.692862 18615 external_mini_cluster.cc:1428] Setting key c046c8fdfb425b6e41347b618bf44cf0
I20260806 22:49:20.692893 16155 raft_consensus.cc:348] T 00000000000000000000000000000000 P 566dbd16450d44bda4b7d6b6e9817ec6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "566dbd16450d44bda4b7d6b6e9817ec6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34011 } }
I20260806 22:49:20.693133 16155 raft_consensus.cc:374] T 00000000000000000000000000000000 P 566dbd16450d44bda4b7d6b6e9817ec6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:20.693177 16155 raft_consensus.cc:729] T 00000000000000000000000000000000 P 566dbd16450d44bda4b7d6b6e9817ec6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 566dbd16450d44bda4b7d6b6e9817ec6, State: Initialized, Role: FOLLOWER
I20260806 22:49:20.693361 16155 consensus_queue.cc:260] T 00000000000000000000000000000000 P 566dbd16450d44bda4b7d6b6e9817ec6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "566dbd16450d44bda4b7d6b6e9817ec6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34011 } }
I20260806 22:49:20.693465 16155 raft_consensus.cc:388] T 00000000000000000000000000000000 P 566dbd16450d44bda4b7d6b6e9817ec6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:49:20.693547 16155 raft_consensus.cc:482] T 00000000000000000000000000000000 P 566dbd16450d44bda4b7d6b6e9817ec6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:49:20.693617 16155 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 566dbd16450d44bda4b7d6b6e9817ec6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:20.694834 16155 raft_consensus.cc:504] T 00000000000000000000000000000000 P 566dbd16450d44bda4b7d6b6e9817ec6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "566dbd16450d44bda4b7d6b6e9817ec6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34011 } }
I20260806 22:49:20.694938 16155 leader_election.cc:302] T 00000000000000000000000000000000 P 566dbd16450d44bda4b7d6b6e9817ec6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 566dbd16450d44bda4b7d6b6e9817ec6; no voters:
I20260806 22:49:20.695158 16155 leader_election.cc:288] T 00000000000000000000000000000000 P 566dbd16450d44bda4b7d6b6e9817ec6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:49:20.695216 16161 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 566dbd16450d44bda4b7d6b6e9817ec6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:49:20.695370 16161 raft_consensus.cc:686] T 00000000000000000000000000000000 P 566dbd16450d44bda4b7d6b6e9817ec6 [term 1 LEADER]: Becoming Leader. State: Replica: 566dbd16450d44bda4b7d6b6e9817ec6, State: Running, Role: LEADER
I20260806 22:49:20.695474 16155 sys_catalog.cc:564] T 00000000000000000000000000000000 P 566dbd16450d44bda4b7d6b6e9817ec6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:49:20.695528 16161 consensus_queue.cc:237] T 00000000000000000000000000000000 P 566dbd16450d44bda4b7d6b6e9817ec6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "566dbd16450d44bda4b7d6b6e9817ec6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34011 } }
I20260806 22:49:20.695966 16161 sys_catalog.cc:455] T 00000000000000000000000000000000 P 566dbd16450d44bda4b7d6b6e9817ec6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 566dbd16450d44bda4b7d6b6e9817ec6. Latest consensus state: current_term: 1 leader_uuid: "566dbd16450d44bda4b7d6b6e9817ec6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "566dbd16450d44bda4b7d6b6e9817ec6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34011 } } }
I20260806 22:49:20.695979 16162 sys_catalog.cc:455] T 00000000000000000000000000000000 P 566dbd16450d44bda4b7d6b6e9817ec6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "566dbd16450d44bda4b7d6b6e9817ec6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "566dbd16450d44bda4b7d6b6e9817ec6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34011 } } }
I20260806 22:49:20.696105 16161 sys_catalog.cc:458] T 00000000000000000000000000000000 P 566dbd16450d44bda4b7d6b6e9817ec6 [sys.catalog]: This master's current role is: LEADER
I20260806 22:49:20.696130 16162 sys_catalog.cc:458] T 00000000000000000000000000000000 P 566dbd16450d44bda4b7d6b6e9817ec6 [sys.catalog]: This master's current role is: LEADER
I20260806 22:49:20.696357 16170 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:49:20.696976 16170 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:49:20.699631 16160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:20.693495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.166:48827 (local address 127.0.0.1:34011)
0806 22:49:20.693732 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:20.693742 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:20.693777 (+ 35us) server_negotiation.cc:407] Connection header received
0806 22:49:20.693925 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:20.693931 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:20.694015 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:20.694150 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:20.694498 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:20.695931 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:20.697180 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:20.697825 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:20.697944 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:20.698763 (+ 819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:20.698769 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:20.698769 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:49:20.699065 (+ 296us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:20.699112 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:49:20.699129 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:20.699218 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:20.699340 (+ 122us) server_negotiation.cc:299] Negotiation successful
0806 22:49:20.699379 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":134,"threads_started":1}
I20260806 22:49:20.700671 16170 catalog_manager.cc:1347] Generated new cluster ID: 878c4cb4cb5e4f409a7d65aae82e586f
I20260806 22:49:20.700731 16170 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:49:20.701174 18615 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:49:20.710282 16160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:20.702965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.166:47319 (local address 127.0.0.1:34011)
0806 22:49:20.703031 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:20.703033 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:20.707243 (+ 4210us) server_negotiation.cc:407] Connection header received
0806 22:49:20.707327 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:20.707328 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:20.707355 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:20.707416 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:20.707596 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:20.708115 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:20.708860 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:20.709090 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:20.709169 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:20.709992 (+ 823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:20.709994 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:20.709994 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:49:20.709999 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:20.710016 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:49:20.710021 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:20.710090 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:20.710153 (+ 63us) server_negotiation.cc:299] Negotiation successful
0806 22:49:20.710168 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 22:49:20.711098 16170 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:49:20.711838 16170 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:49:20.721292 16170 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 566dbd16450d44bda4b7d6b6e9817ec6: Generated new TSK 0
I20260806 22:49:20.721537 16170 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:49:20.731081 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 16056
W20260806 22:49:20.748540 16037 connection.cc:535] client connection to 127.0.0.1:34011 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34011): Connection reset by peer (error 104)
2026-08-06T22:49:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1638 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056275803658-18615-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:49:20 dist-test-slave-h3ft krb5kdc[16199](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:49:20 dist-test-slave-h3ft krb5kdc[16199](info): set up 2 sockets
Aug 06 22:49:20 dist-test-slave-h3ft krb5kdc[16199](info): commencing operation
krb5kdc: starting...
W20260806 22:49:22.060134 18615 mini_kdc.cc:121] Time spent starting KDC: real 1.297s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Aug 06 22:49:22 dist-test-slave-h3ft krb5kdc[16199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056562, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:49:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:49:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056275803658-18615-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056275803658-18615-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:49:22.202688 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056275803658-18615-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056275803658-18615-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056275803658-18615-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37061
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056275803658-18615-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37061
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:49:22.298225 16221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:22.298497 16221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:22.298523 16221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:22.299757 16221 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:49:22.299796 16221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:22.299811 16221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:22.299824 16221 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:49:22.299836 16221 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:49:22.301368 16221 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056275803658-18615-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056275803658-18615-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37061
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056275803658-18615-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37061
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16221
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056275803658-18615-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:49:22.301638 16221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:22.301860 16221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 22:49:22.307660 16221 server_base.cc:1034] running on GCE node
W20260806 22:49:22.307606 16228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:22.307626 16227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:22.307585 16230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:22.308199 16221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:22.308579 16221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:22.309733 16221 hybrid_clock.cc:648] HybridClock initialized: now 1786056562309698 us; error 53 us; skew 500 ppm
Aug 06 22:49:22 dist-test-slave-h3ft krb5kdc[16199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056562, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:22.318464 16221 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:49:22.319141 16221 webserver.cc:460] Webserver started at http://127.0.0.1:37505/ using document root <none> and password file <none>
I20260806 22:49:22.319414 16221 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:22.319487 16221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:22.319660 16221 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:49:22.320962 16221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056275803658-18615-0/minicluster-data/master-0/data/instance:
uuid: "ec36c43153724b869bb66844956db135"
format_stamp: "Formatted at 2026-08-06 22:49:22 on dist-test-slave-h3ft"
server_key: "7d85a6cd7e1ec35d6582cd182923783f"
server_key_iv: "ce7b5a6cc325729f48e1f521d81d6328"
server_key_version: "encryptionkey@0"
I20260806 22:49:22.321435 16221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056275803658-18615-0/minicluster-data/master-0/wal/instance:
uuid: "ec36c43153724b869bb66844956db135"
format_stamp: "Formatted at 2026-08-06 22:49:22 on dist-test-slave-h3ft"
server_key: "7d85a6cd7e1ec35d6582cd182923783f"
server_key_iv: "ce7b5a6cc325729f48e1f521d81d6328"
server_key_version: "encryptionkey@0"
I20260806 22:49:22.323083 16221 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260806 22:49:22.324043 16237 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:22.324270 16221 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:49:22.324393 16221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056275803658-18615-0/minicluster-data/master-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056275803658-18615-0/minicluster-data/master-0/wal
uuid: "ec36c43153724b869bb66844956db135"
format_stamp: "Formatted at 2026-08-06 22:49:22 on dist-test-slave-h3ft"
server_key: "7d85a6cd7e1ec35d6582cd182923783f"
server_key_iv: "ce7b5a6cc325729f48e1f521d81d6328"
server_key_version: "encryptionkey@0"
I20260806 22:49:22.324474 16221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056275803658-18615-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056275803658-18615-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056275803658-18615-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:22.328356 16221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:22.330767 16221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:22.330945 16221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:22.335917 16221 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37061
I20260806 22:49:22.335956 16300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37061 every 8 connection(s)
I20260806 22:49:22.336369 16221 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
I20260806 22:49:22.337531 16302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:22.339861 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 16221
I20260806 22:49:22.339998 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786056275803658-18615-0/minicluster-data/master-0/wal/instance
I20260806 22:49:22.340237 18615 external_mini_cluster.cc:1428] Setting key 57af8ce75434e9774fa8e73203095215
I20260806 22:49:22.341449 16302 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec36c43153724b869bb66844956db135: Bootstrap starting.
I20260806 22:49:22.342285 16302 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec36c43153724b869bb66844956db135: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:22.342664 16302 log.cc:824] T 00000000000000000000000000000000 P ec36c43153724b869bb66844956db135: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:22.343191 16307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:22.340897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.166:36433 (local address 127.0.0.1:37061)
0806 22:49:22.341119 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:22.341129 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:22.341158 (+ 29us) server_negotiation.cc:407] Connection header received
0806 22:49:22.341700 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:22.341709 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:22.341752 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:22.341886 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:22.342692 (+ 806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:22.342697 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:22.342698 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:49:22.342703 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:22.342753 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:49:22.342767 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:22.342873 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:22.342908 (+ 35us) server_negotiation.cc:299] Negotiation successful
0806 22:49:22.342962 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":105,"threads_started":1}
I20260806 22:49:22.343600 16302 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec36c43153724b869bb66844956db135: No bootstrap required, opened a new log
I20260806 22:49:22.344362 18615 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:49:22.346469 16302 raft_consensus.cc:348] T 00000000000000000000000000000000 P ec36c43153724b869bb66844956db135 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec36c43153724b869bb66844956db135" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37061 } }
I20260806 22:49:22.346676 16302 raft_consensus.cc:374] T 00000000000000000000000000000000 P ec36c43153724b869bb66844956db135 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:22.346735 16302 raft_consensus.cc:729] T 00000000000000000000000000000000 P ec36c43153724b869bb66844956db135 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec36c43153724b869bb66844956db135, State: Initialized, Role: FOLLOWER
I20260806 22:49:22.346882 16302 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec36c43153724b869bb66844956db135 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec36c43153724b869bb66844956db135" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37061 } }
I20260806 22:49:22.346992 16302 raft_consensus.cc:388] T 00000000000000000000000000000000 P ec36c43153724b869bb66844956db135 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:49:22.347093 16302 raft_consensus.cc:482] T 00000000000000000000000000000000 P ec36c43153724b869bb66844956db135 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:49:22.347153 16302 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ec36c43153724b869bb66844956db135 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:22.347491 16307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:22.346319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.166:49351 (local address 127.0.0.1:37061)
0806 22:49:22.346376 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:22.346378 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:22.346532 (+ 154us) server_negotiation.cc:407] Connection header received
0806 22:49:22.346617 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:22.346617 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:22.346623 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:22.346685 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:22.347312 (+ 627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:22.347314 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:22.347314 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:49:22.347318 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:22.347332 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:49:22.347336 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:22.347380 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:22.347436 (+ 56us) server_negotiation.cc:299] Negotiation successful
0806 22:49:22.347447 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 22:49:22.347999 16302 raft_consensus.cc:504] T 00000000000000000000000000000000 P ec36c43153724b869bb66844956db135 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec36c43153724b869bb66844956db135" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37061 } }
I20260806 22:49:22.348120 16302 leader_election.cc:302] T 00000000000000000000000000000000 P ec36c43153724b869bb66844956db135 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ec36c43153724b869bb66844956db135; no voters:
I20260806 22:49:22.348282 16302 leader_election.cc:288] T 00000000000000000000000000000000 P ec36c43153724b869bb66844956db135 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:49:22.348322 16319 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ec36c43153724b869bb66844956db135 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:49:22.348490 16319 raft_consensus.cc:686] T 00000000000000000000000000000000 P ec36c43153724b869bb66844956db135 [term 1 LEADER]: Becoming Leader. State: Replica: ec36c43153724b869bb66844956db135, State: Running, Role: LEADER
I20260806 22:49:22.348577 16302 sys_catalog.cc:564] T 00000000000000000000000000000000 P ec36c43153724b869bb66844956db135 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:49:22.348654 16319 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec36c43153724b869bb66844956db135 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec36c43153724b869bb66844956db135" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37061 } }
I20260806 22:49:22.349086 16321 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec36c43153724b869bb66844956db135 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ec36c43153724b869bb66844956db135" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec36c43153724b869bb66844956db135" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37061 } } }
I20260806 22:49:22.349184 16321 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec36c43153724b869bb66844956db135 [sys.catalog]: This master's current role is: LEADER
I20260806 22:49:22.349294 16323 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec36c43153724b869bb66844956db135 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec36c43153724b869bb66844956db135. Latest consensus state: current_term: 1 leader_uuid: "ec36c43153724b869bb66844956db135" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec36c43153724b869bb66844956db135" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37061 } } }
I20260806 22:49:22.349601 16332 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:49:22.349611 16323 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec36c43153724b869bb66844956db135 [sys.catalog]: This master's current role is: LEADER
I20260806 22:49:22.350230 16332 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:49:22.351984 16332 catalog_manager.cc:1347] Generated new cluster ID: effb4468172d489bba4f8a6de4336579
I20260806 22:49:22.352046 16332 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:49:22.361971 16332 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:49:22.362169 16332 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:49:22.364892 16332 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ec36c43153724b869bb66844956db135: Generated new TSK 0
I20260806 22:49:22.365101 16332 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:49:22.367763 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 16221
2026-08-06T22:49:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1640 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056275803658-18615-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:49:22 dist-test-slave-h3ft krb5kdc[16348](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:49:22 dist-test-slave-h3ft krb5kdc[16348](info): set up 2 sockets
Aug 06 22:49:22 dist-test-slave-h3ft krb5kdc[16348](info): commencing operation
krb5kdc: starting...
W20260806 22:49:23.655951 18615 mini_kdc.cc:121] Time spent starting KDC: real 1.252s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Aug 06 22:49:23 dist-test-slave-h3ft krb5kdc[16348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056563, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:49:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:49:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056275803658-18615-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056275803658-18615-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:49:23.798951 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056275803658-18615-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056275803658-18615-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056275803658-18615-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39913
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056275803658-18615-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39913
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:49:23.897857 16364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:23.898205 16364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:23.898234 16364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:23.899562 16364 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:49:23.899612 16364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:23.899627 16364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:23.899641 16364 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:49:23.899652 16364 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:49:23.901240 16364 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056275803658-18615-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056275803658-18615-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39913
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056275803658-18615-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39913
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16364
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056275803658-18615-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:49:23.901494 16364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:23.901723 16364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:23.907310 16376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:23.907382 16381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:23.907315 16378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:23.907982 16364 server_base.cc:1034] running on GCE node
I20260806 22:49:23.908255 16364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:23.908569 16364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:23.909785 16364 hybrid_clock.cc:648] HybridClock initialized: now 1786056563909765 us; error 38 us; skew 500 ppm
Aug 06 22:49:23 dist-test-slave-h3ft krb5kdc[16348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056563, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:23.913125 16364 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:49:23.913677 16364 webserver.cc:460] Webserver started at http://127.0.0.1:34029/ using document root <none> and password file <none>
I20260806 22:49:23.913952 16364 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:23.914029 16364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:23.914198 16364 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:49:23.915580 16364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056275803658-18615-0/minicluster-data/master-0/data/instance:
uuid: "90151dce992b4fad9b177f3b1768c002"
format_stamp: "Formatted at 2026-08-06 22:49:23 on dist-test-slave-h3ft"
server_key: "6604caeaeb014efe7f4053d907814ba8"
server_key_iv: "050da58cb75239217ec20c70f816dc5e"
server_key_version: "encryptionkey@0"
I20260806 22:49:23.916018 16364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056275803658-18615-0/minicluster-data/master-0/wal/instance:
uuid: "90151dce992b4fad9b177f3b1768c002"
format_stamp: "Formatted at 2026-08-06 22:49:23 on dist-test-slave-h3ft"
server_key: "6604caeaeb014efe7f4053d907814ba8"
server_key_iv: "050da58cb75239217ec20c70f816dc5e"
server_key_version: "encryptionkey@0"
I20260806 22:49:23.917603 16364 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:49:23.918452 16392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:23.918641 16364 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:49:23.918766 16364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056275803658-18615-0/minicluster-data/master-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056275803658-18615-0/minicluster-data/master-0/wal
uuid: "90151dce992b4fad9b177f3b1768c002"
format_stamp: "Formatted at 2026-08-06 22:49:23 on dist-test-slave-h3ft"
server_key: "6604caeaeb014efe7f4053d907814ba8"
server_key_iv: "050da58cb75239217ec20c70f816dc5e"
server_key_version: "encryptionkey@0"
I20260806 22:49:23.918857 16364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056275803658-18615-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056275803658-18615-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056275803658-18615-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:23.928893 16364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:23.931823 16364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:23.932034 16364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:23.937495 16364 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39913
I20260806 22:49:23.937546 16461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39913 every 8 connection(s)
I20260806 22:49:23.938031 16364 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
I20260806 22:49:23.939211 16462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:23.942027 16462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90151dce992b4fad9b177f3b1768c002: Bootstrap starting.
I20260806 22:49:23.942716 16462 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 90151dce992b4fad9b177f3b1768c002: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:23.943053 16462 log.cc:824] T 00000000000000000000000000000000 P 90151dce992b4fad9b177f3b1768c002: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:23.943877 16462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90151dce992b4fad9b177f3b1768c002: No bootstrap required, opened a new log
I20260806 22:49:23.945670 16462 raft_consensus.cc:348] T 00000000000000000000000000000000 P 90151dce992b4fad9b177f3b1768c002 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90151dce992b4fad9b177f3b1768c002" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39913 } }
I20260806 22:49:23.945834 16462 raft_consensus.cc:374] T 00000000000000000000000000000000 P 90151dce992b4fad9b177f3b1768c002 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:23.945860 16462 raft_consensus.cc:729] T 00000000000000000000000000000000 P 90151dce992b4fad9b177f3b1768c002 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90151dce992b4fad9b177f3b1768c002, State: Initialized, Role: FOLLOWER
I20260806 22:49:23.946030 16462 consensus_queue.cc:260] T 00000000000000000000000000000000 P 90151dce992b4fad9b177f3b1768c002 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90151dce992b4fad9b177f3b1768c002" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39913 } }
I20260806 22:49:23.946128 16462 raft_consensus.cc:388] T 00000000000000000000000000000000 P 90151dce992b4fad9b177f3b1768c002 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:49:23.946156 16462 raft_consensus.cc:482] T 00000000000000000000000000000000 P 90151dce992b4fad9b177f3b1768c002 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:49:23.946188 16462 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 90151dce992b4fad9b177f3b1768c002 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:23.946759 16462 raft_consensus.cc:504] T 00000000000000000000000000000000 P 90151dce992b4fad9b177f3b1768c002 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90151dce992b4fad9b177f3b1768c002" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39913 } }
I20260806 22:49:23.946859 16462 leader_election.cc:302] T 00000000000000000000000000000000 P 90151dce992b4fad9b177f3b1768c002 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 90151dce992b4fad9b177f3b1768c002; no voters:
I20260806 22:49:23.947067 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 16364
I20260806 22:49:23.947157 16462 leader_election.cc:288] T 00000000000000000000000000000000 P 90151dce992b4fad9b177f3b1768c002 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:49:23.947209 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786056275803658-18615-0/minicluster-data/master-0/wal/instance
I20260806 22:49:23.947182 16465 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 90151dce992b4fad9b177f3b1768c002 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:49:23.947384 16465 raft_consensus.cc:686] T 00000000000000000000000000000000 P 90151dce992b4fad9b177f3b1768c002 [term 1 LEADER]: Becoming Leader. State: Replica: 90151dce992b4fad9b177f3b1768c002, State: Running, Role: LEADER
I20260806 22:49:23.947501 18615 external_mini_cluster.cc:1428] Setting key 4c2ee0c0c12b64d4556a79f32dab6182
I20260806 22:49:23.947486 16465 consensus_queue.cc:237] T 00000000000000000000000000000000 P 90151dce992b4fad9b177f3b1768c002 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90151dce992b4fad9b177f3b1768c002" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39913 } }
I20260806 22:49:23.947572 16462 sys_catalog.cc:564] T 00000000000000000000000000000000 P 90151dce992b4fad9b177f3b1768c002 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:49:23.947834 16465 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90151dce992b4fad9b177f3b1768c002 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90151dce992b4fad9b177f3b1768c002. Latest consensus state: current_term: 1 leader_uuid: "90151dce992b4fad9b177f3b1768c002" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90151dce992b4fad9b177f3b1768c002" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39913 } } }
I20260806 22:49:23.947897 16465 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90151dce992b4fad9b177f3b1768c002 [sys.catalog]: This master's current role is: LEADER
I20260806 22:49:23.947850 16466 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90151dce992b4fad9b177f3b1768c002 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "90151dce992b4fad9b177f3b1768c002" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90151dce992b4fad9b177f3b1768c002" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39913 } } }
I20260806 22:49:23.948015 16466 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90151dce992b4fad9b177f3b1768c002 [sys.catalog]: This master's current role is: LEADER
I20260806 22:49:23.948220 16471 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:49:23.948897 16471 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:49:23.950884 16471 catalog_manager.cc:1347] Generated new cluster ID: 1e586aad14624893a136176071524fbf
I20260806 22:49:23.950948 16471 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:49:23 dist-test-slave-h3ft krb5kdc[16348](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056563, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 22:49:23.956249 16471 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:49:23.956964 16471 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:49:23.960026 16476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:23.948353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.166:58901 (local address 127.0.0.1:39913)
0806 22:49:23.948580 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:23.948594 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:23.948633 (+ 39us) server_negotiation.cc:407] Connection header received
0806 22:49:23.948780 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:23.948788 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:23.948877 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:23.949718 (+ 841us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:23.950274 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:23.951555 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:23.952442 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:23.952965 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:23.953099 (+ 134us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:23.956050 (+ 2951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:23.956055 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:23.956055 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:23.956070 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:23.958126 (+ 2056us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:23.958562 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:23.958566 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:23.958566 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:23.958671 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:23.958962 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:23.958964 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:23.958964 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:23.959253 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:23.959476 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:23.959573 (+ 97us) server_negotiation.cc:299] Negotiation successful
0806 22:49:23.959825 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":116,"threads_started":1}
I20260806 22:49:23.961181 18615 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:49:23.962252 16471 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 90151dce992b4fad9b177f3b1768c002: Generated new TSK 0
I20260806 22:49:23.962423 16471 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:49:23.970585 16476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:23.962965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.166:57627 (local address 127.0.0.1:39913)
0806 22:49:23.963025 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:23.963027 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:23.963156 (+ 129us) server_negotiation.cc:407] Connection header received
0806 22:49:23.963220 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:23.963221 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:23.963234 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:23.963297 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:23.963807 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:23.964663 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:23.965525 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:23.965901 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:23.965978 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:23.967522 (+ 1544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:23.967524 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:23.967524 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:23.967532 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:23.969309 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:23.969707 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:23.969708 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:23.969709 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:23.969778 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:23.970044 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:23.970045 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:23.970046 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:23.970186 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:23.970300 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:23.970409 (+ 109us) server_negotiation.cc:299] Negotiation successful
0806 22:49:23.970476 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 22:49:23.972872 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 16364
W20260806 22:49:23.988641 16346 connection.cc:535] client connection to 127.0.0.1:39913 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39913): Connection reset by peer (error 104)
2026-08-06T22:49:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1599 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056275803658-18615-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:49:24 dist-test-slave-h3ft krb5kdc[16515](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:49:24 dist-test-slave-h3ft krb5kdc[16515](info): set up 2 sockets
Aug 06 22:49:24 dist-test-slave-h3ft krb5kdc[16515](info): commencing operation
krb5kdc: starting...
W20260806 22:49:25.232180 18615 mini_kdc.cc:121] Time spent starting KDC: real 1.229s user 0.002s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Aug 06 22:49:25 dist-test-slave-h3ft krb5kdc[16515](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:49:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:49:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056275803658-18615-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056275803658-18615-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:49:25.377905 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056275803658-18615-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056275803658-18615-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056275803658-18615-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41821
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056275803658-18615-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41821
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:49:25.475530 16536 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:25.475860 16536 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:25.475907 16536 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:25.477315 16536 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:49:25.477399 16536 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:25.477425 16536 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:25.477447 16536 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:49:25.477479 16536 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:49:25.479349 16536 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056275803658-18615-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056275803658-18615-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41821
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056275803658-18615-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41821
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16536
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056275803658-18615-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:49:25.479707 16536 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:25.479996 16536 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:25.485908 16544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:25.485908 16541 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:25.486065 16542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:25.486450 16536 server_base.cc:1034] running on GCE node
I20260806 22:49:25.486730 16536 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:25.487113 16536 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:25.488272 16536 hybrid_clock.cc:648] HybridClock initialized: now 1786056565488250 us; error 50 us; skew 500 ppm
Aug 06 22:49:25 dist-test-slave-h3ft krb5kdc[16515](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056565, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:25.497613 16536 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:49:25.498304 16536 webserver.cc:460] Webserver started at http://127.0.0.1:44821/ using document root <none> and password file <none>
I20260806 22:49:25.498598 16536 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:25.498649 16536 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:25.498783 16536 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:49:25.500234 16536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056275803658-18615-0/minicluster-data/master-0/data/instance:
uuid: "a73aecd1193443f480d6f0b9a10d6505"
format_stamp: "Formatted at 2026-08-06 22:49:25 on dist-test-slave-h3ft"
server_key: "b0c9518df37132be9325410c3386ffd3"
server_key_iv: "fea38138190cde2af2e7647f3a10903c"
server_key_version: "encryptionkey@0"
I20260806 22:49:25.500669 16536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056275803658-18615-0/minicluster-data/master-0/wal/instance:
uuid: "a73aecd1193443f480d6f0b9a10d6505"
format_stamp: "Formatted at 2026-08-06 22:49:25 on dist-test-slave-h3ft"
server_key: "b0c9518df37132be9325410c3386ffd3"
server_key_iv: "fea38138190cde2af2e7647f3a10903c"
server_key_version: "encryptionkey@0"
I20260806 22:49:25.502427 16536 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260806 22:49:25.503564 16555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:25.503752 16536 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:49:25.503870 16536 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056275803658-18615-0/minicluster-data/master-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056275803658-18615-0/minicluster-data/master-0/wal
uuid: "a73aecd1193443f480d6f0b9a10d6505"
format_stamp: "Formatted at 2026-08-06 22:49:25 on dist-test-slave-h3ft"
server_key: "b0c9518df37132be9325410c3386ffd3"
server_key_iv: "fea38138190cde2af2e7647f3a10903c"
server_key_version: "encryptionkey@0"
I20260806 22:49:25.503950 16536 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056275803658-18615-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056275803658-18615-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056275803658-18615-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:25.524319 16536 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:25.527271 16536 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:25.527424 16536 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:25.532536 16536 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41821
I20260806 22:49:25.532637 16627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41821 every 8 connection(s)
I20260806 22:49:25.533092 16536 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
I20260806 22:49:25.534317 16628 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:25.536363 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 16536
I20260806 22:49:25.536530 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786056275803658-18615-0/minicluster-data/master-0/wal/instance
I20260806 22:49:25.536793 18615 external_mini_cluster.cc:1428] Setting key 9ae37ba7d95b1894b90f6b2619acd5f9
I20260806 22:49:25.538002 16628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a73aecd1193443f480d6f0b9a10d6505: Bootstrap starting.
I20260806 22:49:25.538779 16628 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a73aecd1193443f480d6f0b9a10d6505: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:25.539263 16628 log.cc:824] T 00000000000000000000000000000000 P a73aecd1193443f480d6f0b9a10d6505: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:25.540179 16628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a73aecd1193443f480d6f0b9a10d6505: No bootstrap required, opened a new log
I20260806 22:49:25.542758 16628 raft_consensus.cc:348] T 00000000000000000000000000000000 P a73aecd1193443f480d6f0b9a10d6505 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a73aecd1193443f480d6f0b9a10d6505" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41821 } }
I20260806 22:49:25.542953 16628 raft_consensus.cc:374] T 00000000000000000000000000000000 P a73aecd1193443f480d6f0b9a10d6505 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:25.543032 16628 raft_consensus.cc:729] T 00000000000000000000000000000000 P a73aecd1193443f480d6f0b9a10d6505 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a73aecd1193443f480d6f0b9a10d6505, State: Initialized, Role: FOLLOWER
I20260806 22:49:25.543246 16628 consensus_queue.cc:260] T 00000000000000000000000000000000 P a73aecd1193443f480d6f0b9a10d6505 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a73aecd1193443f480d6f0b9a10d6505" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41821 } }
I20260806 22:49:25.543365 16628 raft_consensus.cc:388] T 00000000000000000000000000000000 P a73aecd1193443f480d6f0b9a10d6505 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:49:25.543419 16628 raft_consensus.cc:482] T 00000000000000000000000000000000 P a73aecd1193443f480d6f0b9a10d6505 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:49:25.543473 16628 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a73aecd1193443f480d6f0b9a10d6505 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:25.544041 16631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:25.537574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.166:42559 (local address 127.0.0.1:41821)
0806 22:49:25.537829 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:25.537842 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:25.537883 (+ 41us) server_negotiation.cc:407] Connection header received
0806 22:49:25.538443 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:25.538451 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:25.538525 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:25.538706 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:25.538994 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:25.540498 (+ 1504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:25.541389 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:25.542113 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:25.542287 (+ 174us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:25.543170 (+ 883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:25.543176 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:25.543176 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:49:25.543453 (+ 277us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:25.543504 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:49:25.543523 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:25.543624 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:25.543745 (+ 121us) server_negotiation.cc:299] Negotiation successful
0806 22:49:25.543792 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":117,"threads_started":1}
I20260806 22:49:25.544431 16628 raft_consensus.cc:504] T 00000000000000000000000000000000 P a73aecd1193443f480d6f0b9a10d6505 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a73aecd1193443f480d6f0b9a10d6505" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41821 } }
I20260806 22:49:25.544556 16628 leader_election.cc:302] T 00000000000000000000000000000000 P a73aecd1193443f480d6f0b9a10d6505 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a73aecd1193443f480d6f0b9a10d6505; no voters:
I20260806 22:49:25.544782 16628 leader_election.cc:288] T 00000000000000000000000000000000 P a73aecd1193443f480d6f0b9a10d6505 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:49:25.544926 16633 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a73aecd1193443f480d6f0b9a10d6505 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:49:25.545126 16628 sys_catalog.cc:564] T 00000000000000000000000000000000 P a73aecd1193443f480d6f0b9a10d6505 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:49:25.545254 18615 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:49:25.545277 16633 raft_consensus.cc:686] T 00000000000000000000000000000000 P a73aecd1193443f480d6f0b9a10d6505 [term 1 LEADER]: Becoming Leader. State: Replica: a73aecd1193443f480d6f0b9a10d6505, State: Running, Role: LEADER
I20260806 22:49:25.545377 16633 consensus_queue.cc:237] T 00000000000000000000000000000000 P a73aecd1193443f480d6f0b9a10d6505 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a73aecd1193443f480d6f0b9a10d6505" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41821 } }
I20260806 22:49:25.545832 16636 sys_catalog.cc:455] T 00000000000000000000000000000000 P a73aecd1193443f480d6f0b9a10d6505 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a73aecd1193443f480d6f0b9a10d6505" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a73aecd1193443f480d6f0b9a10d6505" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41821 } } }
I20260806 22:49:25.545931 16636 sys_catalog.cc:458] T 00000000000000000000000000000000 P a73aecd1193443f480d6f0b9a10d6505 [sys.catalog]: This master's current role is: LEADER
I20260806 22:49:25.546253 16637 sys_catalog.cc:455] T 00000000000000000000000000000000 P a73aecd1193443f480d6f0b9a10d6505 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a73aecd1193443f480d6f0b9a10d6505. Latest consensus state: current_term: 1 leader_uuid: "a73aecd1193443f480d6f0b9a10d6505" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a73aecd1193443f480d6f0b9a10d6505" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41821 } } }
I20260806 22:49:25.546317 16637 sys_catalog.cc:458] T 00000000000000000000000000000000 P a73aecd1193443f480d6f0b9a10d6505 [sys.catalog]: This master's current role is: LEADER
I20260806 22:49:25.546413 16646 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:49:25.547116 16646 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:49:25.549163 16646 catalog_manager.cc:1347] Generated new cluster ID: 741655edcfc840a6959d2539563c242f
I20260806 22:49:25.549220 16646 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:49:25.551308 16631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:25.547332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.166:34665 (local address 127.0.0.1:41821)
0806 22:49:25.547371 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:25.547373 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:25.547714 (+ 341us) server_negotiation.cc:407] Connection header received
0806 22:49:25.547820 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:25.547822 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:25.547837 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:25.547870 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:25.548118 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:25.548883 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:25.549756 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:25.550004 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:25.550107 (+ 103us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:25.550958 (+ 851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:25.550960 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:25.550960 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:49:25.550966 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:25.550983 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:49:25.550988 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:25.551068 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:25.551191 (+ 123us) server_negotiation.cc:299] Negotiation successful
0806 22:49:25.551204 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 22:49:25.557847 16646 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:49:25.558429 16646 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:49:25.566253 16646 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a73aecd1193443f480d6f0b9a10d6505: Generated new TSK 0
I20260806 22:49:25.566526 16646 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:49:25.572708 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 16536
W20260806 22:49:25.591733 16513 connection.cc:535] client connection to 127.0.0.1:41821 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41821): Connection reset by peer (error 104)
2026-08-06T22:49:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1603 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056275803658-18615-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:49:25 dist-test-slave-h3ft krb5kdc[16672](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:49:25 dist-test-slave-h3ft krb5kdc[16672](info): set up 2 sockets
Aug 06 22:49:25 dist-test-slave-h3ft krb5kdc[16672](info): commencing operation
krb5kdc: starting...
W20260806 22:49:26.928210 18615 mini_kdc.cc:121] Time spent starting KDC: real 1.321s user 0.001s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Aug 06 22:49:27 dist-test-slave-h3ft krb5kdc[16672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056567, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:49:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:49:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056275803658-18615-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056275803658-18615-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 22:49:27.069847 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056275803658-18615-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056275803658-18615-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056275803658-18615-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42517
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056275803658-18615-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42517
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:49:27.166767 16692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:27.167099 16692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:27.167157 16692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:27.168373 16692 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:49:27.168440 16692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:27.168464 16692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:27.168485 16692 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:49:27.168517 16692 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:49:27.170161 16692 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056275803658-18615-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056275803658-18615-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42517
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056275803658-18615-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42517
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16692
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056275803658-18615-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:49:27.170485 16692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:27.170774 16692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:27.176595 16700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:27.176584 16699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:27.176559 16702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:27.176805 16692 server_base.cc:1034] running on GCE node
I20260806 22:49:27.177227 16692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:27.177551 16692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:27.178784 16692 hybrid_clock.cc:648] HybridClock initialized: now 1786056567178769 us; error 49 us; skew 500 ppm
Aug 06 22:49:27 dist-test-slave-h3ft krb5kdc[16672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056567, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:27.181879 16692 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 22:49:27.182364 16692 webserver.cc:460] Webserver started at http://127.0.0.1:34679/ using document root <none> and password file <none>
I20260806 22:49:27.182610 16692 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:27.182657 16692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:27.182760 16692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:49:27.184051 16692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056275803658-18615-0/minicluster-data/master-0/data/instance:
uuid: "ecc6e5d4e12541f8a520b59a5a6ae9c9"
format_stamp: "Formatted at 2026-08-06 22:49:27 on dist-test-slave-h3ft"
server_key: "b263749a89e9a383908a6beeb8503a34"
server_key_iv: "2f757c65cde61e59a57e13c831e69f46"
server_key_version: "encryptionkey@0"
I20260806 22:49:27.184401 16692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056275803658-18615-0/minicluster-data/master-0/wal/instance:
uuid: "ecc6e5d4e12541f8a520b59a5a6ae9c9"
format_stamp: "Formatted at 2026-08-06 22:49:27 on dist-test-slave-h3ft"
server_key: "b263749a89e9a383908a6beeb8503a34"
server_key_iv: "2f757c65cde61e59a57e13c831e69f46"
server_key_version: "encryptionkey@0"
I20260806 22:49:27.186043 16692 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:49:27.186887 16709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:27.187124 16692 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:49:27.187247 16692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056275803658-18615-0/minicluster-data/master-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056275803658-18615-0/minicluster-data/master-0/wal
uuid: "ecc6e5d4e12541f8a520b59a5a6ae9c9"
format_stamp: "Formatted at 2026-08-06 22:49:27 on dist-test-slave-h3ft"
server_key: "b263749a89e9a383908a6beeb8503a34"
server_key_iv: "2f757c65cde61e59a57e13c831e69f46"
server_key_version: "encryptionkey@0"
I20260806 22:49:27.187340 16692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056275803658-18615-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056275803658-18615-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056275803658-18615-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:27.191169 16692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:27.193742 16692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:27.193919 16692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:27.198706 16692 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42517
I20260806 22:49:27.198770 16788 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42517 every 8 connection(s)
I20260806 22:49:27.199222 16692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
I20260806 22:49:27.200461 16789 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:27.203356 16789 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecc6e5d4e12541f8a520b59a5a6ae9c9: Bootstrap starting.
I20260806 22:49:27.204092 16789 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ecc6e5d4e12541f8a520b59a5a6ae9c9: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:27.204450 16789 log.cc:824] T 00000000000000000000000000000000 P ecc6e5d4e12541f8a520b59a5a6ae9c9: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:27.205302 16789 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecc6e5d4e12541f8a520b59a5a6ae9c9: No bootstrap required, opened a new log
I20260806 22:49:27.207216 16789 raft_consensus.cc:348] T 00000000000000000000000000000000 P ecc6e5d4e12541f8a520b59a5a6ae9c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecc6e5d4e12541f8a520b59a5a6ae9c9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42517 } }
I20260806 22:49:27.207396 16789 raft_consensus.cc:374] T 00000000000000000000000000000000 P ecc6e5d4e12541f8a520b59a5a6ae9c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:27.207443 16789 raft_consensus.cc:729] T 00000000000000000000000000000000 P ecc6e5d4e12541f8a520b59a5a6ae9c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecc6e5d4e12541f8a520b59a5a6ae9c9, State: Initialized, Role: FOLLOWER
I20260806 22:49:27.207665 16789 consensus_queue.cc:260] T 00000000000000000000000000000000 P ecc6e5d4e12541f8a520b59a5a6ae9c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecc6e5d4e12541f8a520b59a5a6ae9c9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42517 } }
I20260806 22:49:27.207726 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 16692
I20260806 22:49:27.207773 16789 raft_consensus.cc:388] T 00000000000000000000000000000000 P ecc6e5d4e12541f8a520b59a5a6ae9c9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:49:27.207813 16789 raft_consensus.cc:482] T 00000000000000000000000000000000 P ecc6e5d4e12541f8a520b59a5a6ae9c9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:49:27.207873 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786056275803658-18615-0/minicluster-data/master-0/wal/instance
I20260806 22:49:27.207872 16789 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ecc6e5d4e12541f8a520b59a5a6ae9c9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:27.208123 18615 external_mini_cluster.cc:1428] Setting key 98495eb0a3c389a9baa041c4927a101e
I20260806 22:49:27.208601 16789 raft_consensus.cc:504] T 00000000000000000000000000000000 P ecc6e5d4e12541f8a520b59a5a6ae9c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecc6e5d4e12541f8a520b59a5a6ae9c9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42517 } }
I20260806 22:49:27.208741 16789 leader_election.cc:302] T 00000000000000000000000000000000 P ecc6e5d4e12541f8a520b59a5a6ae9c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ecc6e5d4e12541f8a520b59a5a6ae9c9; no voters:
I20260806 22:49:27.209133 16789 leader_election.cc:288] T 00000000000000000000000000000000 P ecc6e5d4e12541f8a520b59a5a6ae9c9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:49:27.209184 16795 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ecc6e5d4e12541f8a520b59a5a6ae9c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:49:27.209383 16795 raft_consensus.cc:686] T 00000000000000000000000000000000 P ecc6e5d4e12541f8a520b59a5a6ae9c9 [term 1 LEADER]: Becoming Leader. State: Replica: ecc6e5d4e12541f8a520b59a5a6ae9c9, State: Running, Role: LEADER
I20260806 22:49:27.209507 16789 sys_catalog.cc:564] T 00000000000000000000000000000000 P ecc6e5d4e12541f8a520b59a5a6ae9c9 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:49:27.209544 16795 consensus_queue.cc:237] T 00000000000000000000000000000000 P ecc6e5d4e12541f8a520b59a5a6ae9c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecc6e5d4e12541f8a520b59a5a6ae9c9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42517 } }
I20260806 22:49:27.209973 16795 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecc6e5d4e12541f8a520b59a5a6ae9c9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ecc6e5d4e12541f8a520b59a5a6ae9c9. Latest consensus state: current_term: 1 leader_uuid: "ecc6e5d4e12541f8a520b59a5a6ae9c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecc6e5d4e12541f8a520b59a5a6ae9c9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42517 } } }
I20260806 22:49:27.210119 16795 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecc6e5d4e12541f8a520b59a5a6ae9c9 [sys.catalog]: This master's current role is: LEADER
I20260806 22:49:27.209990 16796 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecc6e5d4e12541f8a520b59a5a6ae9c9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ecc6e5d4e12541f8a520b59a5a6ae9c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecc6e5d4e12541f8a520b59a5a6ae9c9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42517 } } }
I20260806 22:49:27.211024 16796 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecc6e5d4e12541f8a520b59a5a6ae9c9 [sys.catalog]: This master's current role is: LEADER
I20260806 22:49:27.211251 16794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:27.208742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.166:34431 (local address 127.0.0.1:42517)
0806 22:49:27.209000 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:27.209012 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:27.209145 (+ 133us) server_negotiation.cc:407] Connection header received
0806 22:49:27.209337 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:27.209350 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:27.209389 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:27.209595 (+ 206us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:27.210669 (+ 1074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:27.210674 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:27.210674 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:49:27.210681 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:27.210741 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:49:27.210761 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:27.210841 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:27.210931 (+ 90us) server_negotiation.cc:299] Negotiation successful
0806 22:49:27.210988 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":122,"threads_started":1}
I20260806 22:49:27.211972 16800 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:49:27.212409 18615 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:49:27.212639 16800 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:49:27.214514 16800 catalog_manager.cc:1347] Generated new cluster ID: ed8a24dd5a0844c69247966850e5b1a1
I20260806 22:49:27.214571 16800 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:49:27.215768 16794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:27.214198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.166:47417 (local address 127.0.0.1:42517)
0806 22:49:27.214234 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:27.214236 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:27.214610 (+ 374us) server_negotiation.cc:407] Connection header received
0806 22:49:27.214658 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:27.214659 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:27.214667 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:27.214696 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:27.215512 (+ 816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:27.215514 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:27.215514 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:49:27.215519 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:27.215536 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:49:27.215541 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:27.215597 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:27.215659 (+ 62us) server_negotiation.cc:299] Negotiation successful
0806 22:49:27.215677 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 22:49:27.237826 16800 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:49:27.238121 16800 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:49:27.244303 16800 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ecc6e5d4e12541f8a520b59a5a6ae9c9: Generated new TSK 0
I20260806 22:49:27.244560 16800 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:49:27.246686 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 16692
2026-08-06T22:49:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1671 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056275803658-18615-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:49:27 dist-test-slave-h3ft krb5kdc[16831](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:49:27 dist-test-slave-h3ft krb5kdc[16831](info): set up 2 sockets
Aug 06 22:49:27 dist-test-slave-h3ft krb5kdc[16831](info): commencing operation
krb5kdc: starting...
W20260806 22:49:28.520254 18615 mini_kdc.cc:121] Time spent starting KDC: real 1.242s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Aug 06 22:49:28 dist-test-slave-h3ft krb5kdc[16831](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056568, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:49:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:49:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.45.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab.
Entry for principal kudu/127.18.45.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab.
I20260806 22:49:28.666719 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056275803658-18615-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056275803658-18615-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056275803658-18615-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:46851
--webserver_interface=127.18.45.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:46163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.45.254:46851
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:49:28.765095 16849 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:28.765406 16849 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:28.765451 16849 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:28.766724 16849 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:49:28.766808 16849 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:28.766835 16849 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:28.766860 16849 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:49:28.766891 16849 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:49:28.768677 16849 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:46163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056275803658-18615-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056275803658-18615-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.45.254:46851
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:46851
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.45.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16849
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056275803658-18615-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:49:28.769025 16849 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:28.769316 16849 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:28.775295 16859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:28.775449 16849 server_base.cc:1034] running on GCE node
W20260806 22:49:28.775358 16855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:28.775542 16856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:28.775871 16849 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:28.776254 16849 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:28.777436 16849 hybrid_clock.cc:648] HybridClock initialized: now 1786056568777413 us; error 51 us; skew 500 ppm
Aug 06 22:49:28 dist-test-slave-h3ft krb5kdc[16831](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056568, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:28.780501 16849 init.cc:377] Logged in from keytab as kudu/127.18.45.254@KRBTEST.COM (short username kudu)
I20260806 22:49:28.781085 16849 webserver.cc:460] Webserver started at http://127.18.45.254:40353/ using document root <none> and password file <none>
I20260806 22:49:28.781381 16849 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:28.781458 16849 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:28.781628 16849 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:49:28.782872 16849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056275803658-18615-0/minicluster-data/master-0/data/instance:
uuid: "328fbedc99dc4a379b8973dd1dc68f56"
format_stamp: "Formatted at 2026-08-06 22:49:28 on dist-test-slave-h3ft"
server_key: "5499337b8831e91bc760b5592d303e53"
server_key_iv: "a6f6d42442517a692fa76a1bea695595"
server_key_version: "encryptionkey@0"
I20260806 22:49:28.783353 16849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056275803658-18615-0/minicluster-data/master-0/wal/instance:
uuid: "328fbedc99dc4a379b8973dd1dc68f56"
format_stamp: "Formatted at 2026-08-06 22:49:28 on dist-test-slave-h3ft"
server_key: "5499337b8831e91bc760b5592d303e53"
server_key_iv: "a6f6d42442517a692fa76a1bea695595"
server_key_version: "encryptionkey@0"
I20260806 22:49:28.784876 16849 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 22:49:28.785691 16866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:28.785876 16849 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:49:28.785995 16849 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056275803658-18615-0/minicluster-data/master-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056275803658-18615-0/minicluster-data/master-0/wal
uuid: "328fbedc99dc4a379b8973dd1dc68f56"
format_stamp: "Formatted at 2026-08-06 22:49:28 on dist-test-slave-h3ft"
server_key: "5499337b8831e91bc760b5592d303e53"
server_key_iv: "a6f6d42442517a692fa76a1bea695595"
server_key_version: "encryptionkey@0"
I20260806 22:49:28.786082 16849 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056275803658-18615-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056275803658-18615-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056275803658-18615-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:28.797602 16849 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:28.800364 16849 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:28.800565 16849 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:28.805670 16849 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.254:46851
I20260806 22:49:28.805724 16945 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.254:46851 every 8 connection(s)
I20260806 22:49:28.806172 16849 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
I20260806 22:49:28.807267 16946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:28.809907 16946 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 328fbedc99dc4a379b8973dd1dc68f56: Bootstrap starting.
I20260806 22:49:28.810580 16946 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 328fbedc99dc4a379b8973dd1dc68f56: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:28.810863 16946 log.cc:824] T 00000000000000000000000000000000 P 328fbedc99dc4a379b8973dd1dc68f56: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:28.811683 16946 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 328fbedc99dc4a379b8973dd1dc68f56: No bootstrap required, opened a new log
I20260806 22:49:28.813519 16946 raft_consensus.cc:348] T 00000000000000000000000000000000 P 328fbedc99dc4a379b8973dd1dc68f56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "328fbedc99dc4a379b8973dd1dc68f56" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 46851 } }
I20260806 22:49:28.813688 16946 raft_consensus.cc:374] T 00000000000000000000000000000000 P 328fbedc99dc4a379b8973dd1dc68f56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:28.813714 16946 raft_consensus.cc:729] T 00000000000000000000000000000000 P 328fbedc99dc4a379b8973dd1dc68f56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 328fbedc99dc4a379b8973dd1dc68f56, State: Initialized, Role: FOLLOWER
I20260806 22:49:28.813882 16946 consensus_queue.cc:260] T 00000000000000000000000000000000 P 328fbedc99dc4a379b8973dd1dc68f56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "328fbedc99dc4a379b8973dd1dc68f56" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 46851 } }
I20260806 22:49:28.814013 16946 raft_consensus.cc:388] T 00000000000000000000000000000000 P 328fbedc99dc4a379b8973dd1dc68f56 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:49:28.814067 16946 raft_consensus.cc:482] T 00000000000000000000000000000000 P 328fbedc99dc4a379b8973dd1dc68f56 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:49:28.814133 16946 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 328fbedc99dc4a379b8973dd1dc68f56 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:28.814821 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 16849
I20260806 22:49:28.814993 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786056275803658-18615-0/minicluster-data/master-0/wal/instance
I20260806 22:49:28.815291 18615 external_mini_cluster.cc:1428] Setting key 7eb31951a21bc331ed4a9f73071a1479
I20260806 22:49:28.815508 16946 raft_consensus.cc:504] T 00000000000000000000000000000000 P 328fbedc99dc4a379b8973dd1dc68f56 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "328fbedc99dc4a379b8973dd1dc68f56" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 46851 } }
I20260806 22:49:28.815649 16946 leader_election.cc:302] T 00000000000000000000000000000000 P 328fbedc99dc4a379b8973dd1dc68f56 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 328fbedc99dc4a379b8973dd1dc68f56; no voters:
I20260806 22:49:28.815856 16946 leader_election.cc:288] T 00000000000000000000000000000000 P 328fbedc99dc4a379b8973dd1dc68f56 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:49:28.816110 16950 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 328fbedc99dc4a379b8973dd1dc68f56 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:49:28.816208 16946 sys_catalog.cc:564] T 00000000000000000000000000000000 P 328fbedc99dc4a379b8973dd1dc68f56 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:49:28.816246 16950 raft_consensus.cc:686] T 00000000000000000000000000000000 P 328fbedc99dc4a379b8973dd1dc68f56 [term 1 LEADER]: Becoming Leader. State: Replica: 328fbedc99dc4a379b8973dd1dc68f56, State: Running, Role: LEADER
I20260806 22:49:28.816437 16950 consensus_queue.cc:237] T 00000000000000000000000000000000 P 328fbedc99dc4a379b8973dd1dc68f56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "328fbedc99dc4a379b8973dd1dc68f56" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 46851 } }
I20260806 22:49:28.816926 16950 sys_catalog.cc:455] T 00000000000000000000000000000000 P 328fbedc99dc4a379b8973dd1dc68f56 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 328fbedc99dc4a379b8973dd1dc68f56. Latest consensus state: current_term: 1 leader_uuid: "328fbedc99dc4a379b8973dd1dc68f56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "328fbedc99dc4a379b8973dd1dc68f56" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 46851 } } }
I20260806 22:49:28.817088 16950 sys_catalog.cc:458] T 00000000000000000000000000000000 P 328fbedc99dc4a379b8973dd1dc68f56 [sys.catalog]: This master's current role is: LEADER
I20260806 22:49:28.817384 16963 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:49:28.818091 16963 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:49:28.818049 16954 sys_catalog.cc:455] T 00000000000000000000000000000000 P 328fbedc99dc4a379b8973dd1dc68f56 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "328fbedc99dc4a379b8973dd1dc68f56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "328fbedc99dc4a379b8973dd1dc68f56" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 46851 } } }
I20260806 22:49:28.818240 16954 sys_catalog.cc:458] T 00000000000000000000000000000000 P 328fbedc99dc4a379b8973dd1dc68f56 [sys.catalog]: This master's current role is: LEADER
I20260806 22:49:28.820039 16963 catalog_manager.cc:1347] Generated new cluster ID: 6d34a7c7d1eb4777b66ded8e9950c5f5
I20260806 22:49:28.820098 16963 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:49:28 dist-test-slave-h3ft krb5kdc[16831](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056568, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:49:28.827668 16953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:28.815935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58819 (local address 127.18.45.254:46851)
0806 22:49:28.816357 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:28.816369 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:28.816410 (+ 41us) server_negotiation.cc:407] Connection header received
0806 22:49:28.816560 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:28.816569 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:28.816640 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:28.816827 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:28.817816 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:28.819393 (+ 1577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:28.820430 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:28.820918 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:28.821063 (+ 145us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:28.823956 (+ 2893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:28.823960 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:28.823961 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:28.823975 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:28.825842 (+ 1867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:28.826282 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:28.826291 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:28.826292 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:28.826408 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:28.826669 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:28.826671 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:28.826671 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:28.826953 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:28.827195 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:28.827309 (+ 114us) server_negotiation.cc:299] Negotiation successful
0806 22:49:28.827431 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":80,"threads_started":1}
I20260806 22:49:28.828814 18615 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:49:28.829721 16963 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:49:28.830453 16963 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:49:28.837759 16953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:28.830758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53241 (local address 127.18.45.254:46851)
0806 22:49:28.830816 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:28.830818 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:28.830963 (+ 145us) server_negotiation.cc:407] Connection header received
0806 22:49:28.831080 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:28.831081 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:28.831092 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:28.831150 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:28.831737 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:28.832292 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:28.833079 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:28.833333 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:28.833413 (+ 80us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:28.834793 (+ 1380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:28.834795 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:28.834795 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:28.834803 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:28.836436 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:28.836838 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:28.836840 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:28.836840 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:28.836908 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:28.837153 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:28.837154 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:28.837155 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:28.837299 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:28.837425 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:28.837563 (+ 138us) server_negotiation.cc:299] Negotiation successful
0806 22:49:28.837635 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 22:49:28.838963 16963 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 328fbedc99dc4a379b8973dd1dc68f56: Generated new TSK 0
I20260806 22:49:28.839212 16963 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:49:28.841863 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 16849
W20260806 22:49:28.854885 16828 connection.cc:535] client connection to 127.18.45.254:46851 recv error: Network error: failed to read from TLS socket (remote: 127.18.45.254:46851): Connection reset by peer (error 104)
2026-08-06T22:49:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1591 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056275803658-18615-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:49:28 dist-test-slave-h3ft krb5kdc[16987](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:49:28 dist-test-slave-h3ft krb5kdc[16987](info): set up 2 sockets
Aug 06 22:49:28 dist-test-slave-h3ft krb5kdc[16987](info): commencing operation
krb5kdc: starting...
W20260806 22:49:30.100124 18615 mini_kdc.cc:121] Time spent starting KDC: real 1.230s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Aug 06 22:49:30 dist-test-slave-h3ft krb5kdc[16987](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056570, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:49:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:49:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.45.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab.
Entry for principal kudu/127.18.45.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab.
I20260806 22:49:30.242544 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056275803658-18615-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056275803658-18615-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056275803658-18615-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:37783
--webserver_interface=127.18.45.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:42151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.45.254:37783
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:49:30.337266 17006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:30.337509 17006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:30.337532 17006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:30.338651 17006 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:49:30.338686 17006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:30.338699 17006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:30.338712 17006 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:49:30.338724 17006 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:49:30.340220 17006 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:42151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056275803658-18615-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056275803658-18615-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.45.254:37783
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:37783
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.45.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17006
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056275803658-18615-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:49:30.340456 17006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:30.340667 17006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:30.346467 17012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:30.346496 17018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:30.346462 17014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:30.346760 17006 server_base.cc:1034] running on GCE node
I20260806 22:49:30.347232 17006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:30.347616 17006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:30.348821 17006 hybrid_clock.cc:648] HybridClock initialized: now 1786056570348812 us; error 43 us; skew 500 ppm
Aug 06 22:49:30 dist-test-slave-h3ft krb5kdc[16987](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056570, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:30.351986 17006 init.cc:377] Logged in from keytab as kudu/127.18.45.254@KRBTEST.COM (short username kudu)
I20260806 22:49:30.352520 17006 webserver.cc:460] Webserver started at http://127.18.45.254:43243/ using document root <none> and password file <none>
I20260806 22:49:30.352813 17006 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:30.352895 17006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:30.353057 17006 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:49:30.354379 17006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056275803658-18615-0/minicluster-data/master-0/data/instance:
uuid: "fb660c937a1c440c877302724be5427b"
format_stamp: "Formatted at 2026-08-06 22:49:30 on dist-test-slave-h3ft"
server_key: "5ecbe0f6140bf85fcf7880fd186230b4"
server_key_iv: "125f99afa6c49e7fc85dbd6725e298a5"
server_key_version: "encryptionkey@0"
I20260806 22:49:30.354799 17006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056275803658-18615-0/minicluster-data/master-0/wal/instance:
uuid: "fb660c937a1c440c877302724be5427b"
format_stamp: "Formatted at 2026-08-06 22:49:30 on dist-test-slave-h3ft"
server_key: "5ecbe0f6140bf85fcf7880fd186230b4"
server_key_iv: "125f99afa6c49e7fc85dbd6725e298a5"
server_key_version: "encryptionkey@0"
I20260806 22:49:30.356403 17006 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:49:30.357282 17028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:30.357481 17006 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:49:30.357600 17006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056275803658-18615-0/minicluster-data/master-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056275803658-18615-0/minicluster-data/master-0/wal
uuid: "fb660c937a1c440c877302724be5427b"
format_stamp: "Formatted at 2026-08-06 22:49:30 on dist-test-slave-h3ft"
server_key: "5ecbe0f6140bf85fcf7880fd186230b4"
server_key_iv: "125f99afa6c49e7fc85dbd6725e298a5"
server_key_version: "encryptionkey@0"
I20260806 22:49:30.357684 17006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056275803658-18615-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056275803658-18615-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056275803658-18615-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:30.365185 17006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:30.367257 17006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:30.367439 17006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:30.372430 17006 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.254:37783
I20260806 22:49:30.372470 17093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.254:37783 every 8 connection(s)
I20260806 22:49:30.372928 17006 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
I20260806 22:49:30.373994 17094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:30.376796 17094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb660c937a1c440c877302724be5427b: Bootstrap starting.
I20260806 22:49:30.377521 17094 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb660c937a1c440c877302724be5427b: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:30.377878 17094 log.cc:824] T 00000000000000000000000000000000 P fb660c937a1c440c877302724be5427b: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:30.378742 17094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb660c937a1c440c877302724be5427b: No bootstrap required, opened a new log
I20260806 22:49:30.379693 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 17006
I20260806 22:49:30.379804 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786056275803658-18615-0/minicluster-data/master-0/wal/instance
I20260806 22:49:30.380170 18615 external_mini_cluster.cc:1428] Setting key 74e1cadc3e21d275e552aad732481a9e
I20260806 22:49:30.381006 17094 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb660c937a1c440c877302724be5427b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb660c937a1c440c877302724be5427b" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 37783 } }
I20260806 22:49:30.381217 17094 raft_consensus.cc:374] T 00000000000000000000000000000000 P fb660c937a1c440c877302724be5427b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:30.381263 17094 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb660c937a1c440c877302724be5427b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb660c937a1c440c877302724be5427b, State: Initialized, Role: FOLLOWER
I20260806 22:49:30.381410 17094 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb660c937a1c440c877302724be5427b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb660c937a1c440c877302724be5427b" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 37783 } }
I20260806 22:49:30.381498 17094 raft_consensus.cc:388] T 00000000000000000000000000000000 P fb660c937a1c440c877302724be5427b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:49:30.381528 17094 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb660c937a1c440c877302724be5427b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:49:30.381589 17094 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb660c937a1c440c877302724be5427b [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:30.382917 17094 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb660c937a1c440c877302724be5427b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb660c937a1c440c877302724be5427b" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 37783 } }
I20260806 22:49:30.383080 17094 leader_election.cc:302] T 00000000000000000000000000000000 P fb660c937a1c440c877302724be5427b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb660c937a1c440c877302724be5427b; no voters:
I20260806 22:49:30.383256 17094 leader_election.cc:288] T 00000000000000000000000000000000 P fb660c937a1c440c877302724be5427b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:49:30.383370 17100 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb660c937a1c440c877302724be5427b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:49:30.383579 17100 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb660c937a1c440c877302724be5427b [term 1 LEADER]: Becoming Leader. State: Replica: fb660c937a1c440c877302724be5427b, State: Running, Role: LEADER
I20260806 22:49:30.383623 17094 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb660c937a1c440c877302724be5427b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:49:30.383725 17100 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb660c937a1c440c877302724be5427b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb660c937a1c440c877302724be5427b" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 37783 } }
I20260806 22:49:30.384287 17104 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb660c937a1c440c877302724be5427b [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb660c937a1c440c877302724be5427b. Latest consensus state: current_term: 1 leader_uuid: "fb660c937a1c440c877302724be5427b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb660c937a1c440c877302724be5427b" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 37783 } } }
I20260806 22:49:30.384259 17102 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb660c937a1c440c877302724be5427b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fb660c937a1c440c877302724be5427b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb660c937a1c440c877302724be5427b" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 37783 } } }
I20260806 22:49:30.384373 17104 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb660c937a1c440c877302724be5427b [sys.catalog]: This master's current role is: LEADER
I20260806 22:49:30.384375 17102 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb660c937a1c440c877302724be5427b [sys.catalog]: This master's current role is: LEADER
I20260806 22:49:30.384811 17115 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:49:30.385771 17115 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:49:30.387611 17099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:30.380955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53631 (local address 127.18.45.254:37783)
0806 22:49:30.381216 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:30.381227 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:30.381268 (+ 41us) server_negotiation.cc:407] Connection header received
0806 22:49:30.381405 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:30.381413 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:30.381479 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:30.381657 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:30.381951 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:30.383522 (+ 1571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:30.384856 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:30.385573 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:30.385713 (+ 140us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:30.386616 (+ 903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:30.386621 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:30.386621 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:49:30.386833 (+ 212us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:30.386871 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:49:30.386884 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:30.386989 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:30.387117 (+ 128us) server_negotiation.cc:299] Negotiation successful
0806 22:49:30.387349 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":128,"threads_started":1}
I20260806 22:49:30.387779 17115 catalog_manager.cc:1347] Generated new cluster ID: 9e83dc0a81764d1d8ec20571e69faa6f
I20260806 22:49:30.387959 17115 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:49:30.388913 18615 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:49:30.393735 17115 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:49:30.394300 17115 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:49:30.394308 17099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:30.390566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54057 (local address 127.18.45.254:37783)
0806 22:49:30.390684 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:30.390686 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:30.390734 (+ 48us) server_negotiation.cc:407] Connection header received
0806 22:49:30.390813 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:30.390814 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:30.390826 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:30.390883 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:30.391095 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:30.391595 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:30.392358 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:30.392952 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:30.393040 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:30.393970 (+ 930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:30.393972 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:30.393972 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:49:30.393977 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:30.393997 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:49:30.394004 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:30.394039 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:30.394183 (+ 144us) server_negotiation.cc:299] Negotiation successful
0806 22:49:30.394222 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":81}
I20260806 22:49:30.398087 17115 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fb660c937a1c440c877302724be5427b: Generated new TSK 0
I20260806 22:49:30.398304 17115 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:49:30.402585 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 17006
W20260806 22:49:30.416419 16982 connection.cc:535] client connection to 127.18.45.254:37783 recv error: Network error: failed to read from TLS socket (remote: 127.18.45.254:37783): Connection reset by peer (error 104)
2026-08-06T22:49:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1561 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056275803658-18615-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:49:30 dist-test-slave-h3ft krb5kdc[17138](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:49:30 dist-test-slave-h3ft krb5kdc[17138](info): set up 2 sockets
Aug 06 22:49:30 dist-test-slave-h3ft krb5kdc[17138](info): commencing operation
krb5kdc: starting...
W20260806 22:49:31.675714 18615 mini_kdc.cc:121] Time spent starting KDC: real 1.245s user 0.003s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Aug 06 22:49:31 dist-test-slave-h3ft krb5kdc[17138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056571, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:49:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:49:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.45.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab.
Entry for principal kudu/127.18.45.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab.
I20260806 22:49:31.818585 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056275803658-18615-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056275803658-18615-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056275803658-18615-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:46573
--webserver_interface=127.18.45.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:46779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.45.254:46573
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:49:31.914767 17156 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:31.915148 17156 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:31.915201 17156 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:31.916716 17156 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:49:31.916815 17156 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:31.916843 17156 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:31.916873 17156 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:49:31.916921 17156 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:49:31.918646 17156 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:46779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056275803658-18615-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056275803658-18615-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.45.254:46573
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:46573
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.45.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17156
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056275803658-18615-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:49:31.919034 17156 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:31.919324 17156 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:31.925011 17163 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:31.925042 17162 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:31.925045 17165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:31.925428 17156 server_base.cc:1034] running on GCE node
I20260806 22:49:31.925740 17156 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:31.926047 17156 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:31.927176 17156 hybrid_clock.cc:648] HybridClock initialized: now 1786056571927160 us; error 29 us; skew 500 ppm
Aug 06 22:49:31 dist-test-slave-h3ft krb5kdc[17138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056571, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:31.939983 17156 init.cc:377] Logged in from keytab as kudu/127.18.45.254@KRBTEST.COM (short username kudu)
I20260806 22:49:31.940653 17156 webserver.cc:460] Webserver started at http://127.18.45.254:44491/ using document root <none> and password file <none>
I20260806 22:49:31.940932 17156 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:31.941010 17156 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:31.941187 17156 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:49:31.942471 17156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056275803658-18615-0/minicluster-data/master-0/data/instance:
uuid: "8c949f03d99d487ea1a62a9c6091f870"
format_stamp: "Formatted at 2026-08-06 22:49:31 on dist-test-slave-h3ft"
server_key: "b0037926269b2d1bccd332911c4abdec"
server_key_iv: "1a01148699dbf6757ea5e3238bc88848"
server_key_version: "encryptionkey@0"
I20260806 22:49:31.942902 17156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056275803658-18615-0/minicluster-data/master-0/wal/instance:
uuid: "8c949f03d99d487ea1a62a9c6091f870"
format_stamp: "Formatted at 2026-08-06 22:49:31 on dist-test-slave-h3ft"
server_key: "b0037926269b2d1bccd332911c4abdec"
server_key_iv: "1a01148699dbf6757ea5e3238bc88848"
server_key_version: "encryptionkey@0"
I20260806 22:49:31.944514 17156 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:49:31.945437 17172 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:31.945665 17156 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:49:31.945789 17156 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056275803658-18615-0/minicluster-data/master-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056275803658-18615-0/minicluster-data/master-0/wal
uuid: "8c949f03d99d487ea1a62a9c6091f870"
format_stamp: "Formatted at 2026-08-06 22:49:31 on dist-test-slave-h3ft"
server_key: "b0037926269b2d1bccd332911c4abdec"
server_key_iv: "1a01148699dbf6757ea5e3238bc88848"
server_key_version: "encryptionkey@0"
I20260806 22:49:31.945873 17156 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056275803658-18615-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056275803658-18615-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056275803658-18615-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:31.949774 17156 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:31.952286 17156 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:31.952469 17156 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:31.957659 17156 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.254:46573
I20260806 22:49:31.957716 17242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.254:46573 every 8 connection(s)
I20260806 22:49:31.958138 17156 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
I20260806 22:49:31.959467 17244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:31.962308 17244 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c949f03d99d487ea1a62a9c6091f870: Bootstrap starting.
I20260806 22:49:31.963088 17244 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8c949f03d99d487ea1a62a9c6091f870: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:31.963397 17244 log.cc:824] T 00000000000000000000000000000000 P 8c949f03d99d487ea1a62a9c6091f870: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:31.964190 17244 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c949f03d99d487ea1a62a9c6091f870: No bootstrap required, opened a new log
I20260806 22:49:31.965586 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 17156
I20260806 22:49:31.965746 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786056275803658-18615-0/minicluster-data/master-0/wal/instance
I20260806 22:49:31.965975 18615 external_mini_cluster.cc:1428] Setting key 9a29530c0cb10731e6f918bb366097c6
I20260806 22:49:31.966424 17244 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8c949f03d99d487ea1a62a9c6091f870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c949f03d99d487ea1a62a9c6091f870" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 46573 } }
I20260806 22:49:31.966653 17244 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8c949f03d99d487ea1a62a9c6091f870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:31.966745 17244 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8c949f03d99d487ea1a62a9c6091f870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c949f03d99d487ea1a62a9c6091f870, State: Initialized, Role: FOLLOWER
I20260806 22:49:31.966962 17244 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c949f03d99d487ea1a62a9c6091f870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c949f03d99d487ea1a62a9c6091f870" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 46573 } }
I20260806 22:49:31.967126 17244 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8c949f03d99d487ea1a62a9c6091f870 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:49:31.967181 17244 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8c949f03d99d487ea1a62a9c6091f870 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:49:31.967237 17244 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8c949f03d99d487ea1a62a9c6091f870 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:31.968127 17244 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8c949f03d99d487ea1a62a9c6091f870 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c949f03d99d487ea1a62a9c6091f870" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 46573 } }
I20260806 22:49:31.968259 17244 leader_election.cc:302] T 00000000000000000000000000000000 P 8c949f03d99d487ea1a62a9c6091f870 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8c949f03d99d487ea1a62a9c6091f870; no voters:
I20260806 22:49:31.968474 17244 leader_election.cc:288] T 00000000000000000000000000000000 P 8c949f03d99d487ea1a62a9c6091f870 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:49:31.968523 17250 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8c949f03d99d487ea1a62a9c6091f870 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:49:31.968726 17250 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8c949f03d99d487ea1a62a9c6091f870 [term 1 LEADER]: Becoming Leader. State: Replica: 8c949f03d99d487ea1a62a9c6091f870, State: Running, Role: LEADER
I20260806 22:49:31.968885 17249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:31.966649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50053 (local address 127.18.45.254:46573)
0806 22:49:31.966875 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:31.966885 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:31.966915 (+ 30us) server_negotiation.cc:407] Connection header received
0806 22:49:31.967123 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:31.967131 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:31.967179 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:31.967339 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:31.968303 (+ 964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:31.968307 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:31.968307 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:49:31.968312 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:31.968366 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:49:31.968382 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:31.968471 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:31.968566 (+ 95us) server_negotiation.cc:299] Negotiation successful
0806 22:49:31.968619 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":118,"threads_started":1}
I20260806 22:49:31.968881 17244 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8c949f03d99d487ea1a62a9c6091f870 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:49:31.968895 17250 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8c949f03d99d487ea1a62a9c6091f870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c949f03d99d487ea1a62a9c6091f870" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 46573 } }
I20260806 22:49:31.970073 18615 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:49:31.970688 17251 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c949f03d99d487ea1a62a9c6091f870 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8c949f03d99d487ea1a62a9c6091f870" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c949f03d99d487ea1a62a9c6091f870" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 46573 } } }
I20260806 22:49:31.970813 17251 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c949f03d99d487ea1a62a9c6091f870 [sys.catalog]: This master's current role is: LEADER
I20260806 22:49:31.971132 17253 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c949f03d99d487ea1a62a9c6091f870 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c949f03d99d487ea1a62a9c6091f870. Latest consensus state: current_term: 1 leader_uuid: "8c949f03d99d487ea1a62a9c6091f870" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c949f03d99d487ea1a62a9c6091f870" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 46573 } } }
I20260806 22:49:31.972335 17253 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c949f03d99d487ea1a62a9c6091f870 [sys.catalog]: This master's current role is: LEADER
W20260806 22:49:31.971417 17273 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8c949f03d99d487ea1a62a9c6091f870: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:49:31.972455 17273 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:49:31.972522 17274 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:49:31.972754 17274 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:49:31.973423 17249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:31.972037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59519 (local address 127.18.45.254:46573)
0806 22:49:31.972072 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:31.972074 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:31.972274 (+ 200us) server_negotiation.cc:407] Connection header received
0806 22:49:31.972358 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:31.972359 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:31.972365 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:31.972397 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:31.973162 (+ 765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:31.973164 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:31.973164 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:49:31.973168 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:31.973188 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:49:31.973194 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:31.973261 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:31.973340 (+ 79us) server_negotiation.cc:299] Negotiation successful
0806 22:49:31.973353 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 22:49:31.975063 17274 catalog_manager.cc:1347] Generated new cluster ID: 7e42f57f241d4f46bb59a4a4083f755c
I20260806 22:49:31.975126 17274 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:49:31.981257 17274 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:49:31.981459 17274 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:49:31.984090 17274 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8c949f03d99d487ea1a62a9c6091f870: Generated new TSK 0
I20260806 22:49:31.984299 17274 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:49:31.988227 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 17156
2026-08-06T22:49:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1597 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056275803658-18615-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:49:32 dist-test-slave-h3ft krb5kdc[17287](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:49:32 dist-test-slave-h3ft krb5kdc[17287](info): set up 2 sockets
Aug 06 22:49:32 dist-test-slave-h3ft krb5kdc[17287](info): commencing operation
krb5kdc: starting...
W20260806 22:49:33.269750 18615 mini_kdc.cc:121] Time spent starting KDC: real 1.242s user 0.008s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Aug 06 22:49:33 dist-test-slave-h3ft krb5kdc[17287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056573, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:49:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:49:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.45.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab.
Entry for principal kudu/127.18.45.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab.
I20260806 22:49:33.405819 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056275803658-18615-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056275803658-18615-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056275803658-18615-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:44417
--webserver_interface=127.18.45.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:36373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.45.254:44417
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:49:33.500798 17304 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:33.501128 17304 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:33.501173 17304 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:33.502435 17304 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:49:33.502517 17304 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:33.502540 17304 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:33.502580 17304 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:49:33.502609 17304 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:49:33.504364 17304 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:36373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056275803658-18615-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056275803658-18615-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.45.254:44417
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:44417
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.45.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17304
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056275803658-18615-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:49:33.504709 17304 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:33.504987 17304 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:33.510768 17310 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:33.510763 17314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:33.510768 17312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:33.511098 17304 server_base.cc:1034] running on GCE node
I20260806 22:49:33.511425 17304 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:33.511850 17304 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:33.513020 17304 hybrid_clock.cc:648] HybridClock initialized: now 1786056573512997 us; error 46 us; skew 500 ppm
Aug 06 22:49:33 dist-test-slave-h3ft krb5kdc[17287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056573, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:33.516177 17304 init.cc:377] Logged in from keytab as kudu/127.18.45.254@KRBTEST.COM (short username kudu)
I20260806 22:49:33.516705 17304 webserver.cc:460] Webserver started at http://127.18.45.254:41539/ using document root <none> and password file <none>
I20260806 22:49:33.517014 17304 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:33.517069 17304 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:33.517261 17304 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:49:33.518589 17304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056275803658-18615-0/minicluster-data/master-0/data/instance:
uuid: "b50664fa982b492fb01dc975a268fba3"
format_stamp: "Formatted at 2026-08-06 22:49:33 on dist-test-slave-h3ft"
server_key: "c41ff3d3c20f7f89eaff187423939b1b"
server_key_iv: "e5236dd35ed3fdf5cbda2c27892be8a4"
server_key_version: "encryptionkey@0"
I20260806 22:49:33.519075 17304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056275803658-18615-0/minicluster-data/master-0/wal/instance:
uuid: "b50664fa982b492fb01dc975a268fba3"
format_stamp: "Formatted at 2026-08-06 22:49:33 on dist-test-slave-h3ft"
server_key: "c41ff3d3c20f7f89eaff187423939b1b"
server_key_iv: "e5236dd35ed3fdf5cbda2c27892be8a4"
server_key_version: "encryptionkey@0"
I20260806 22:49:33.520619 17304 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:49:33.521448 17321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:33.521651 17304 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:49:33.521747 17304 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056275803658-18615-0/minicluster-data/master-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056275803658-18615-0/minicluster-data/master-0/wal
uuid: "b50664fa982b492fb01dc975a268fba3"
format_stamp: "Formatted at 2026-08-06 22:49:33 on dist-test-slave-h3ft"
server_key: "c41ff3d3c20f7f89eaff187423939b1b"
server_key_iv: "e5236dd35ed3fdf5cbda2c27892be8a4"
server_key_version: "encryptionkey@0"
I20260806 22:49:33.521827 17304 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056275803658-18615-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056275803658-18615-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056275803658-18615-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:33.536275 17304 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:33.538642 17304 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:33.538836 17304 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:33.544306 17304 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.254:44417
I20260806 22:49:33.544423 17396 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.254:44417 every 8 connection(s)
I20260806 22:49:33.544862 17304 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
I20260806 22:49:33.546123 17399 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:33.549050 17399 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b50664fa982b492fb01dc975a268fba3: Bootstrap starting.
I20260806 22:49:33.549799 17399 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b50664fa982b492fb01dc975a268fba3: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:33.550159 17399 log.cc:824] T 00000000000000000000000000000000 P b50664fa982b492fb01dc975a268fba3: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:33.551039 17399 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b50664fa982b492fb01dc975a268fba3: No bootstrap required, opened a new log
I20260806 22:49:33.552863 17399 raft_consensus.cc:348] T 00000000000000000000000000000000 P b50664fa982b492fb01dc975a268fba3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b50664fa982b492fb01dc975a268fba3" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 44417 } }
I20260806 22:49:33.553050 17399 raft_consensus.cc:374] T 00000000000000000000000000000000 P b50664fa982b492fb01dc975a268fba3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:33.553107 17399 raft_consensus.cc:729] T 00000000000000000000000000000000 P b50664fa982b492fb01dc975a268fba3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b50664fa982b492fb01dc975a268fba3, State: Initialized, Role: FOLLOWER
I20260806 22:49:33.553196 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 17304
I20260806 22:49:33.553361 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786056275803658-18615-0/minicluster-data/master-0/wal/instance
I20260806 22:49:33.553303 17399 consensus_queue.cc:260] T 00000000000000000000000000000000 P b50664fa982b492fb01dc975a268fba3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b50664fa982b492fb01dc975a268fba3" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 44417 } }
I20260806 22:49:33.553400 17399 raft_consensus.cc:388] T 00000000000000000000000000000000 P b50664fa982b492fb01dc975a268fba3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:49:33.553471 17399 raft_consensus.cc:482] T 00000000000000000000000000000000 P b50664fa982b492fb01dc975a268fba3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:49:33.553525 17399 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b50664fa982b492fb01dc975a268fba3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:33.553589 18615 external_mini_cluster.cc:1428] Setting key ee35d9f9e82555a3c0d5325e09b9b131
I20260806 22:49:33.554883 17399 raft_consensus.cc:504] T 00000000000000000000000000000000 P b50664fa982b492fb01dc975a268fba3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b50664fa982b492fb01dc975a268fba3" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 44417 } }
I20260806 22:49:33.555025 17399 leader_election.cc:302] T 00000000000000000000000000000000 P b50664fa982b492fb01dc975a268fba3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b50664fa982b492fb01dc975a268fba3; no voters:
I20260806 22:49:33.555198 17399 leader_election.cc:288] T 00000000000000000000000000000000 P b50664fa982b492fb01dc975a268fba3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:49:33.555222 17405 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b50664fa982b492fb01dc975a268fba3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:49:33.555351 17405 raft_consensus.cc:686] T 00000000000000000000000000000000 P b50664fa982b492fb01dc975a268fba3 [term 1 LEADER]: Becoming Leader. State: Replica: b50664fa982b492fb01dc975a268fba3, State: Running, Role: LEADER
I20260806 22:49:33.555519 17399 sys_catalog.cc:564] T 00000000000000000000000000000000 P b50664fa982b492fb01dc975a268fba3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:49:33.555522 17405 consensus_queue.cc:237] T 00000000000000000000000000000000 P b50664fa982b492fb01dc975a268fba3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b50664fa982b492fb01dc975a268fba3" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 44417 } }
I20260806 22:49:33.556023 17407 sys_catalog.cc:455] T 00000000000000000000000000000000 P b50664fa982b492fb01dc975a268fba3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b50664fa982b492fb01dc975a268fba3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b50664fa982b492fb01dc975a268fba3" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 44417 } } }
I20260806 22:49:33.556139 17407 sys_catalog.cc:458] T 00000000000000000000000000000000 P b50664fa982b492fb01dc975a268fba3 [sys.catalog]: This master's current role is: LEADER
I20260806 22:49:33.556432 17408 sys_catalog.cc:455] T 00000000000000000000000000000000 P b50664fa982b492fb01dc975a268fba3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b50664fa982b492fb01dc975a268fba3. Latest consensus state: current_term: 1 leader_uuid: "b50664fa982b492fb01dc975a268fba3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b50664fa982b492fb01dc975a268fba3" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 44417 } } }
I20260806 22:49:33.556555 17408 sys_catalog.cc:458] T 00000000000000000000000000000000 P b50664fa982b492fb01dc975a268fba3 [sys.catalog]: This master's current role is: LEADER
I20260806 22:49:33.556727 17413 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:49:33.557350 17413 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:49:33.559567 17413 catalog_manager.cc:1347] Generated new cluster ID: 8e53ec7c24ce47a18f4e5909cc2889c3
I20260806 22:49:33.559626 17413 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 22:49:33 dist-test-slave-h3ft krb5kdc[17287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056573, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:49:33.566731 17404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:33.554230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43609 (local address 127.18.45.254:44417)
0806 22:49:33.554502 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:33.554516 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:33.554561 (+ 45us) server_negotiation.cc:407] Connection header received
0806 22:49:33.554695 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:33.554703 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:33.554768 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:33.554943 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:33.555865 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:33.557549 (+ 1684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:33.558655 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:33.559242 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:33.559372 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:33.562417 (+ 3045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:33.562421 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:33.562422 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:33.562435 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:33.564694 (+ 2259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:33.565074 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:33.565077 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:33.565078 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:33.565196 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:33.565372 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:33.565372 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:33.565373 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:33.565684 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:33.565848 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:33.566330 (+ 482us) server_negotiation.cc:299] Negotiation successful
0806 22:49:33.566485 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":120,"threads_started":1}
I20260806 22:49:33.568001 18615 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:49:33.577397 17404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:33.569854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53377 (local address 127.18.45.254:44417)
0806 22:49:33.569896 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:33.569898 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:33.570042 (+ 144us) server_negotiation.cc:407] Connection header received
0806 22:49:33.570102 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:33.570103 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:33.570115 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:33.570167 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:33.570835 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:33.571605 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:33.572661 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:33.572965 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:33.573025 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:33.574511 (+ 1486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:33.574513 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:33.574513 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:33.574521 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:33.576188 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:33.576605 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:33.576606 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:33.576607 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:33.576666 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:33.576870 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:33.576871 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:33.576871 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:33.576996 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:33.577103 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:33.577225 (+ 122us) server_negotiation.cc:299] Negotiation successful
0806 22:49:33.577300 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:49:33.596218 17413 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:49:33.596864 17413 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:49:33.606091 17413 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b50664fa982b492fb01dc975a268fba3: Generated new TSK 0
I20260806 22:49:33.606377 17413 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:49:33.609124 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 17304
2026-08-06T22:49:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1606 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056275803658-18615-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:49:33 dist-test-slave-h3ft krb5kdc[17436](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:49:33 dist-test-slave-h3ft krb5kdc[17436](info): set up 2 sockets
Aug 06 22:49:33 dist-test-slave-h3ft krb5kdc[17436](info): commencing operation
krb5kdc: starting...
W20260806 22:49:34.869483 18615 mini_kdc.cc:121] Time spent starting KDC: real 1.234s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Aug 06 22:49:34 dist-test-slave-h3ft krb5kdc[17436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056574, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:49:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:49:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.45.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab.
Entry for principal kudu/127.18.45.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab.
I20260806 22:49:35.014775 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056275803658-18615-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056275803658-18615-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056275803658-18615-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:40123
--webserver_interface=127.18.45.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:43913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.45.254:40123
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:49:35.112637 17454 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:35.112928 17454 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:35.112957 17454 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:35.114259 17454 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:49:35.114315 17454 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:35.114336 17454 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:35.114351 17454 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:49:35.114363 17454 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:49:35.116089 17454 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:43913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056275803658-18615-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056275803658-18615-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.45.254:40123
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:40123
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.45.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17454
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056275803658-18615-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:49:35.116382 17454 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:35.116624 17454 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:35.122695 17459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:35.122860 17454 server_base.cc:1034] running on GCE node
W20260806 22:49:35.122704 17460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:35.122704 17463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:35.123299 17454 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:35.123696 17454 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:35.124904 17454 hybrid_clock.cc:648] HybridClock initialized: now 1786056575124875 us; error 54 us; skew 500 ppm
Aug 06 22:49:35 dist-test-slave-h3ft krb5kdc[17436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056575, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:35.128261 17454 init.cc:377] Logged in from keytab as kudu/127.18.45.254@KRBTEST.COM (short username kudu)
I20260806 22:49:35.128778 17454 webserver.cc:460] Webserver started at http://127.18.45.254:42329/ using document root <none> and password file <none>
I20260806 22:49:35.129112 17454 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:35.129168 17454 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:35.129323 17454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:49:35.130666 17454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056275803658-18615-0/minicluster-data/master-0/data/instance:
uuid: "0306beaefc4e4577ab8d31883bbf416f"
format_stamp: "Formatted at 2026-08-06 22:49:35 on dist-test-slave-h3ft"
server_key: "4fdd11f4bb3d6d66bd718f881c19aa4b"
server_key_iv: "bc92998dd45d3c410e8dc68110dd6885"
server_key_version: "encryptionkey@0"
I20260806 22:49:35.131124 17454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056275803658-18615-0/minicluster-data/master-0/wal/instance:
uuid: "0306beaefc4e4577ab8d31883bbf416f"
format_stamp: "Formatted at 2026-08-06 22:49:35 on dist-test-slave-h3ft"
server_key: "4fdd11f4bb3d6d66bd718f881c19aa4b"
server_key_iv: "bc92998dd45d3c410e8dc68110dd6885"
server_key_version: "encryptionkey@0"
I20260806 22:49:35.132766 17454 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:49:35.133790 17476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:35.134091 17454 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:49:35.134297 17454 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056275803658-18615-0/minicluster-data/master-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056275803658-18615-0/minicluster-data/master-0/wal
uuid: "0306beaefc4e4577ab8d31883bbf416f"
format_stamp: "Formatted at 2026-08-06 22:49:35 on dist-test-slave-h3ft"
server_key: "4fdd11f4bb3d6d66bd718f881c19aa4b"
server_key_iv: "bc92998dd45d3c410e8dc68110dd6885"
server_key_version: "encryptionkey@0"
I20260806 22:49:35.134384 17454 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056275803658-18615-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056275803658-18615-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056275803658-18615-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:35.157132 17454 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:35.159655 17454 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:35.159793 17454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:35.164966 17454 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.254:40123
I20260806 22:49:35.165043 17546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.254:40123 every 8 connection(s)
I20260806 22:49:35.165462 17454 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
I20260806 22:49:35.166697 17547 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:35.169780 17547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0306beaefc4e4577ab8d31883bbf416f: Bootstrap starting.
I20260806 22:49:35.170696 17547 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0306beaefc4e4577ab8d31883bbf416f: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:35.171150 17547 log.cc:824] T 00000000000000000000000000000000 P 0306beaefc4e4577ab8d31883bbf416f: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:35.172065 17547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0306beaefc4e4577ab8d31883bbf416f: No bootstrap required, opened a new log
I20260806 22:49:35.173030 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 17454
I20260806 22:49:35.173146 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786056275803658-18615-0/minicluster-data/master-0/wal/instance
I20260806 22:49:35.173390 18615 external_mini_cluster.cc:1428] Setting key 65f73bde9117474c975ba5a236338061
I20260806 22:49:35.174671 17547 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0306beaefc4e4577ab8d31883bbf416f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0306beaefc4e4577ab8d31883bbf416f" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 40123 } }
I20260806 22:49:35.174899 17547 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0306beaefc4e4577ab8d31883bbf416f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:35.174947 17547 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0306beaefc4e4577ab8d31883bbf416f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0306beaefc4e4577ab8d31883bbf416f, State: Initialized, Role: FOLLOWER
I20260806 22:49:35.175140 17547 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0306beaefc4e4577ab8d31883bbf416f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0306beaefc4e4577ab8d31883bbf416f" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 40123 } }
I20260806 22:49:35.175386 17547 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0306beaefc4e4577ab8d31883bbf416f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:49:35.175484 17547 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0306beaefc4e4577ab8d31883bbf416f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:49:35.175551 17547 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0306beaefc4e4577ab8d31883bbf416f [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:35.176275 17547 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0306beaefc4e4577ab8d31883bbf416f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0306beaefc4e4577ab8d31883bbf416f" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 40123 } }
I20260806 22:49:35.176375 17547 leader_election.cc:302] T 00000000000000000000000000000000 P 0306beaefc4e4577ab8d31883bbf416f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0306beaefc4e4577ab8d31883bbf416f; no voters:
I20260806 22:49:35.176617 17547 leader_election.cc:288] T 00000000000000000000000000000000 P 0306beaefc4e4577ab8d31883bbf416f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:49:35.176641 17554 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0306beaefc4e4577ab8d31883bbf416f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:49:35.176772 17554 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0306beaefc4e4577ab8d31883bbf416f [term 1 LEADER]: Becoming Leader. State: Replica: 0306beaefc4e4577ab8d31883bbf416f, State: Running, Role: LEADER
I20260806 22:49:35.176930 17554 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0306beaefc4e4577ab8d31883bbf416f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0306beaefc4e4577ab8d31883bbf416f" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 40123 } }
I20260806 22:49:35.176975 17547 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0306beaefc4e4577ab8d31883bbf416f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:49:35.178292 17561 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0306beaefc4e4577ab8d31883bbf416f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0306beaefc4e4577ab8d31883bbf416f. Latest consensus state: current_term: 1 leader_uuid: "0306beaefc4e4577ab8d31883bbf416f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0306beaefc4e4577ab8d31883bbf416f" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 40123 } } }
I20260806 22:49:35.178411 17561 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0306beaefc4e4577ab8d31883bbf416f [sys.catalog]: This master's current role is: LEADER
I20260806 22:49:35.178691 17555 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0306beaefc4e4577ab8d31883bbf416f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0306beaefc4e4577ab8d31883bbf416f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0306beaefc4e4577ab8d31883bbf416f" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 40123 } } }
I20260806 22:49:35.178653 17576 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:49:35.178846 17555 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0306beaefc4e4577ab8d31883bbf416f [sys.catalog]: This master's current role is: LEADER
I20260806 22:49:35.179435 17576 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:49:35.180775 17553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:35.174087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46957 (local address 127.18.45.254:40123)
0806 22:49:35.174316 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:35.174325 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:35.174355 (+ 30us) server_negotiation.cc:407] Connection header received
0806 22:49:35.174616 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:35.174622 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:35.174673 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:35.174802 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:35.175223 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:35.176790 (+ 1567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:35.178061 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:35.178925 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:35.179062 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:35.179954 (+ 892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:35.179958 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:35.179959 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:49:35.180209 (+ 250us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:35.180259 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:49:35.180272 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:35.180367 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:35.180463 (+ 96us) server_negotiation.cc:299] Negotiation successful
0806 22:49:35.180537 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":125,"threads_started":1}
I20260806 22:49:35.181818 17576 catalog_manager.cc:1347] Generated new cluster ID: e029b1c2d7554146b1d9e98cd00b1d99
I20260806 22:49:35.181875 17576 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:49:35.181967 18615 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:49:35.190328 17553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:35.183627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54849 (local address 127.18.45.254:40123)
0806 22:49:35.183675 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:35.183677 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:35.187259 (+ 3582us) server_negotiation.cc:407] Connection header received
0806 22:49:35.187358 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:35.187359 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:35.187373 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:35.187425 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:35.187603 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:35.188113 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:35.188890 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:35.189107 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:35.189187 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:35.190010 (+ 823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:35.190012 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:35.190013 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:49:35.190018 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:35.190038 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:49:35.190043 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:35.190118 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:35.190180 (+ 62us) server_negotiation.cc:299] Negotiation successful
0806 22:49:35.190208 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:49:35.197243 17576 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:49:35.197903 17576 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:49:35.206961 17576 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0306beaefc4e4577ab8d31883bbf416f: Generated new TSK 0
I20260806 22:49:35.207305 17576 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:49:35.213428 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 17454
2026-08-06T22:49:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1607 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056275803658-18615-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:49:35 dist-test-slave-h3ft krb5kdc[17590](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:49:35 dist-test-slave-h3ft krb5kdc[17590](info): set up 2 sockets
Aug 06 22:49:35 dist-test-slave-h3ft krb5kdc[17590](info): commencing operation
krb5kdc: starting...
W20260806 22:49:36.511350 18615 mini_kdc.cc:121] Time spent starting KDC: real 1.268s user 0.003s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Aug 06 22:49:36 dist-test-slave-h3ft krb5kdc[17590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056576, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:49:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:49:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.45.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab.
Entry for principal kudu/127.18.45.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab.
I20260806 22:49:36.650743 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056275803658-18615-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056275803658-18615-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056275803658-18615-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:41733
--webserver_interface=127.18.45.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:43569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.45.254:41733
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:49:36.748487 17609 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:36.748787 17609 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:36.748831 17609 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:36.750077 17609 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:49:36.750151 17609 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:36.750175 17609 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:36.750198 17609 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:49:36.750229 17609 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:49:36.751989 17609 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:43569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056275803658-18615-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056275803658-18615-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.45.254:41733
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:41733
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.45.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17609
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056275803658-18615-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:49:36.752333 17609 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:36.752635 17609 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:36.758481 17619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:36.758467 17614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:36.758541 17615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:36.758723 17609 server_base.cc:1034] running on GCE node
I20260806 22:49:36.759169 17609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:36.759553 17609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:36.760728 17609 hybrid_clock.cc:648] HybridClock initialized: now 1786056576760709 us; error 41 us; skew 500 ppm
Aug 06 22:49:36 dist-test-slave-h3ft krb5kdc[17590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056576, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:36.763762 17609 init.cc:377] Logged in from keytab as kudu/127.18.45.254@KRBTEST.COM (short username kudu)
I20260806 22:49:36.764245 17609 webserver.cc:460] Webserver started at http://127.18.45.254:39689/ using document root <none> and password file <none>
I20260806 22:49:36.764458 17609 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:36.764503 17609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:36.764600 17609 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:49:36.765790 17609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056275803658-18615-0/minicluster-data/master-0/data/instance:
uuid: "2c1c8383d3b747089c7deb8ccb0a53bd"
format_stamp: "Formatted at 2026-08-06 22:49:36 on dist-test-slave-h3ft"
server_key: "a1f5c7e1d980e038dc699e4b90735d0f"
server_key_iv: "39f1b85472d9d834291456d22d67c66e"
server_key_version: "encryptionkey@0"
I20260806 22:49:36.766150 17609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056275803658-18615-0/minicluster-data/master-0/wal/instance:
uuid: "2c1c8383d3b747089c7deb8ccb0a53bd"
format_stamp: "Formatted at 2026-08-06 22:49:36 on dist-test-slave-h3ft"
server_key: "a1f5c7e1d980e038dc699e4b90735d0f"
server_key_iv: "39f1b85472d9d834291456d22d67c66e"
server_key_version: "encryptionkey@0"
I20260806 22:49:36.767711 17609 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 22:49:36.768610 17628 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:36.768837 17609 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:49:36.768939 17609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056275803658-18615-0/minicluster-data/master-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056275803658-18615-0/minicluster-data/master-0/wal
uuid: "2c1c8383d3b747089c7deb8ccb0a53bd"
format_stamp: "Formatted at 2026-08-06 22:49:36 on dist-test-slave-h3ft"
server_key: "a1f5c7e1d980e038dc699e4b90735d0f"
server_key_iv: "39f1b85472d9d834291456d22d67c66e"
server_key_version: "encryptionkey@0"
I20260806 22:49:36.768999 17609 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056275803658-18615-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056275803658-18615-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056275803658-18615-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:36.772609 17609 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:36.775307 17609 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:36.775502 17609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:36.780143 17609 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.254:41733
I20260806 22:49:36.780257 17698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.254:41733 every 8 connection(s)
I20260806 22:49:36.780583 17609 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
I20260806 22:49:36.781857 17699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:36.784795 17699 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c1c8383d3b747089c7deb8ccb0a53bd: Bootstrap starting.
I20260806 22:49:36.785475 17699 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2c1c8383d3b747089c7deb8ccb0a53bd: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:36.785770 17699 log.cc:824] T 00000000000000000000000000000000 P 2c1c8383d3b747089c7deb8ccb0a53bd: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:36.786595 17699 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c1c8383d3b747089c7deb8ccb0a53bd: No bootstrap required, opened a new log
I20260806 22:49:36.787904 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 17609
I20260806 22:49:36.788053 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786056275803658-18615-0/minicluster-data/master-0/wal/instance
I20260806 22:49:36.788300 18615 external_mini_cluster.cc:1428] Setting key 8bdfedcbf3aaca12f643b461ba597725
I20260806 22:49:36.789088 17699 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2c1c8383d3b747089c7deb8ccb0a53bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c1c8383d3b747089c7deb8ccb0a53bd" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 41733 } }
I20260806 22:49:36.789283 17699 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2c1c8383d3b747089c7deb8ccb0a53bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 22:49:36.789312 17699 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2c1c8383d3b747089c7deb8ccb0a53bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c1c8383d3b747089c7deb8ccb0a53bd, State: Initialized, Role: FOLLOWER
I20260806 22:49:36.789480 17699 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2c1c8383d3b747089c7deb8ccb0a53bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c1c8383d3b747089c7deb8ccb0a53bd" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 41733 } }
I20260806 22:49:36.789579 17699 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2c1c8383d3b747089c7deb8ccb0a53bd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 22:49:36.789657 17699 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2c1c8383d3b747089c7deb8ccb0a53bd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 22:49:36.789727 17699 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2c1c8383d3b747089c7deb8ccb0a53bd [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:36.790551 17699 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2c1c8383d3b747089c7deb8ccb0a53bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c1c8383d3b747089c7deb8ccb0a53bd" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 41733 } }
I20260806 22:49:36.790694 17699 leader_election.cc:302] T 00000000000000000000000000000000 P 2c1c8383d3b747089c7deb8ccb0a53bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2c1c8383d3b747089c7deb8ccb0a53bd; no voters:
I20260806 22:49:36.790915 17699 leader_election.cc:288] T 00000000000000000000000000000000 P 2c1c8383d3b747089c7deb8ccb0a53bd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 22:49:36.790937 17704 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2c1c8383d3b747089c7deb8ccb0a53bd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:49:36.791060 17703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:36.788978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49713 (local address 127.18.45.254:41733)
0806 22:49:36.789197 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:36.789206 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:36.789236 (+ 30us) server_negotiation.cc:407] Connection header received
0806 22:49:36.789411 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:36.789419 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:36.789468 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:36.789638 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:36.790494 (+ 856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:36.790498 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:36.790499 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:49:36.790502 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:36.790541 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:49:36.790561 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:36.790620 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:36.790736 (+ 116us) server_negotiation.cc:299] Negotiation successful
0806 22:49:36.790791 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":123,"threads_started":1}
I20260806 22:49:36.791126 17704 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2c1c8383d3b747089c7deb8ccb0a53bd [term 1 LEADER]: Becoming Leader. State: Replica: 2c1c8383d3b747089c7deb8ccb0a53bd, State: Running, Role: LEADER
I20260806 22:49:36.791286 17699 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2c1c8383d3b747089c7deb8ccb0a53bd [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:49:36.791294 17704 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2c1c8383d3b747089c7deb8ccb0a53bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c1c8383d3b747089c7deb8ccb0a53bd" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 41733 } }
I20260806 22:49:36.791750 17704 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c1c8383d3b747089c7deb8ccb0a53bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2c1c8383d3b747089c7deb8ccb0a53bd. Latest consensus state: current_term: 1 leader_uuid: "2c1c8383d3b747089c7deb8ccb0a53bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c1c8383d3b747089c7deb8ccb0a53bd" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 41733 } } }
I20260806 22:49:36.791922 17704 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c1c8383d3b747089c7deb8ccb0a53bd [sys.catalog]: This master's current role is: LEADER
I20260806 22:49:36.792394 17705 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c1c8383d3b747089c7deb8ccb0a53bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2c1c8383d3b747089c7deb8ccb0a53bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c1c8383d3b747089c7deb8ccb0a53bd" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 41733 } } }
I20260806 22:49:36.792588 17705 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c1c8383d3b747089c7deb8ccb0a53bd [sys.catalog]: This master's current role is: LEADER
I20260806 22:49:36.792611 18615 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260806 22:49:36.795068 17730 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2c1c8383d3b747089c7deb8ccb0a53bd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:49:36.795145 17730 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:49:36.795464 17731 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:49:36.795755 17731 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:49:36.795862 17703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:36.794285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47939 (local address 127.18.45.254:41733)
0806 22:49:36.794330 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:36.794332 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:36.794502 (+ 170us) server_negotiation.cc:407] Connection header received
0806 22:49:36.794708 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:36.794709 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:36.794718 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:36.794747 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:36.795570 (+ 823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:36.795572 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:36.795572 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 22:49:36.795577 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:36.795598 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 22:49:36.795605 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:36.795682 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:36.795736 (+ 54us) server_negotiation.cc:299] Negotiation successful
0806 22:49:36.795751 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:49:36.797719 17731 catalog_manager.cc:1347] Generated new cluster ID: 0727b4b8a8d543189d0caab46ebc56df
I20260806 22:49:36.797766 17731 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:49:36.806927 17731 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:49:36.807245 17731 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:49:36.812500 17731 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2c1c8383d3b747089c7deb8ccb0a53bd: Generated new TSK 0
I20260806 22:49:36.812743 17731 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 22:49:36.815181 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 17609
2026-08-06T22:49:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1600 ms)
[----------] 18 tests from AuthTokenIssuingTest (29193 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:49:36 dist-test-slave-h3ft krb5kdc[17741](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:49:36 dist-test-slave-h3ft krb5kdc[17741](info): set up 2 sockets
Aug 06 22:49:36 dist-test-slave-h3ft krb5kdc[17741](info): commencing operation
krb5kdc: starting...
W20260806 22:49:38.064384 18615 mini_kdc.cc:121] Time spent starting KDC: real 1.220s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Aug 06 22:49:38 dist-test-slave-h3ft krb5kdc[17741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056578, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:49:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:49:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.45.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab.
Entry for principal kudu/127.18.45.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab.
I20260806 22:49:38.213862 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:44487
--webserver_interface=127.18.45.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:46497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.45.254:44487,127.18.45.253:38197,127.18.45.252:46495
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:49:38.308504 17756 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 22:49:38.308780 17756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:38.308799 17756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:38.308820 17756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:38.309966 17756 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:49:38.310003 17756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:38.310019 17756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:38.310031 17756 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:49:38.310043 17756 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:49:38.311647 17756 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:46497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.45.254:44487,127.18.45.253:38197,127.18.45.252:46495
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:44487
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.45.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17756
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:49:38.311898 17756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:38.312111 17756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:38.317776 17762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:38.317811 17766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:38.317859 17763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:38.318032 17756 server_base.cc:1034] running on GCE node
I20260806 22:49:38.318302 17756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:38.318653 17756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:38.319833 17756 hybrid_clock.cc:648] HybridClock initialized: now 1786056578319812 us; error 37 us; skew 500 ppm
Aug 06 22:49:38 dist-test-slave-h3ft krb5kdc[17741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056578, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:38.326843 17756 init.cc:377] Logged in from keytab as kudu/127.18.45.254@KRBTEST.COM (short username kudu)
I20260806 22:49:38.327395 17756 webserver.cc:460] Webserver started at http://127.18.45.254:37783/ using document root <none> and password file <none>
I20260806 22:49:38.327652 17756 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:38.327699 17756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:38.327802 17756 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:49:38.329023 17756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/minicluster-data/master-0/data/instance:
uuid: "64ab4b7cbb194edf8487645652a9954e"
format_stamp: "Formatted at 2026-08-06 22:49:38 on dist-test-slave-h3ft"
server_key: "e9d5ba5bcc8637a6e1bc980f561b939f"
server_key_iv: "f20eb9e6f3792343a7b6a80eb93105bf"
server_key_version: "encryptionkey@0"
I20260806 22:49:38.329391 17756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/minicluster-data/master-0/wal/instance:
uuid: "64ab4b7cbb194edf8487645652a9954e"
format_stamp: "Formatted at 2026-08-06 22:49:38 on dist-test-slave-h3ft"
server_key: "e9d5ba5bcc8637a6e1bc980f561b939f"
server_key_iv: "f20eb9e6f3792343a7b6a80eb93105bf"
server_key_version: "encryptionkey@0"
I20260806 22:49:38.330866 17756 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:49:38.331755 17773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:38.331969 17756 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:49:38.332073 17756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/minicluster-data/master-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/minicluster-data/master-0/wal
uuid: "64ab4b7cbb194edf8487645652a9954e"
format_stamp: "Formatted at 2026-08-06 22:49:38 on dist-test-slave-h3ft"
server_key: "e9d5ba5bcc8637a6e1bc980f561b939f"
server_key_iv: "f20eb9e6f3792343a7b6a80eb93105bf"
server_key_version: "encryptionkey@0"
I20260806 22:49:38.332140 17756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:38.361810 17756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:38.364171 17756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:38.364306 17756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:38.369273 17756 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.254:44487
I20260806 22:49:38.369347 17847 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.254:44487 every 8 connection(s)
I20260806 22:49:38.369772 17756 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
I20260806 22:49:38.371066 17849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:38.371210 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 17756
I20260806 22:49:38.371346 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/minicluster-data/master-0/wal/instance
I20260806 22:49:38.371634 18615 external_mini_cluster.cc:1428] Setting key c3ff9071e6ac1d8ccb96b2257c31b9b5
I20260806 22:49:38.373993 17849 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.45.254" port: 44487 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:49:38 dist-test-slave-h3ft krb5kdc[17741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056578, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.254@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.45.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.253@KRBTEST.COM" created.
I20260806 22:49:38.385923 17854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:38.374934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:44487 (local address 127.0.0.1:42772)
0806 22:49:38.375204 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:38.375256 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:38.375812 (+ 556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:38.376154 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:38.376157 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:38.376449 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:38.376634 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:38.376638 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:38.378293 (+ 1655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:38.378294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:38.379062 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:38.379065 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:38.379863 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:38.379865 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:38.379949 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:38.380589 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:49:38.380599 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:38.382156 (+ 1557us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:38.384326 (+ 2170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:38.384334 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:38.384337 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:38.384536 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:38.384787 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:38.384787 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:38.384788 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:38.384872 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:38.385375 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:38.385384 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:38.385473 (+ 89us) client_negotiation.cc:769] Sending connection context
0806 22:49:38.385570 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 22:49:38.385720 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":178,"threads_started":1}
I20260806 22:49:38.386036 17855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:38.375251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42772 (local address 127.18.45.254:44487)
0806 22:49:38.375428 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:38.375438 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:38.375485 (+ 47us) server_negotiation.cc:407] Connection header received
0806 22:49:38.375987 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:38.375990 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:38.376015 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:38.376137 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:38.376738 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:38.378192 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:38.379141 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:38.379726 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:38.379867 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:38.382248 (+ 2381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:38.382256 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:38.382257 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:38.382265 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:38.384223 (+ 1958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:38.384622 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:38.384625 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:38.384626 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:38.384717 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:38.385001 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:38.385002 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:38.385002 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:38.385301 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:38.385380 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:38.385648 (+ 268us) server_negotiation.cc:299] Negotiation successful
0806 22:49:38.385822 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":136,"thread_start_us":86,"threads_started":1}
I20260806 22:49:38.386842 17849 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.45.253" port: 38197 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:49:38.387274 17854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:38.387153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.253:38197 (local address 127.0.0.1:35322)
0806 22:49:38.387201 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:38.387253 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.253:38197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260806 22:49:38.387444 17776 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.45.253:38197: connect: Connection refused (error 111)
I20260806 22:49:38.387912 17854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:38.387817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.253:38197 (local address 127.0.0.1:35332)
0806 22:49:38.387859 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:38.387884 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.253:38197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260806 22:49:38.388051 17849 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.45.253:38197: Network error: Client connection negotiation failed: client connection to 127.18.45.253:38197: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.253.keytab.
Entry for principal kudu/127.18.45.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.253.keytab.
I20260806 22:49:38.402951 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.253:38197
--webserver_interface=127.18.45.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.253.keytab
--principal=kudu/127.18.45.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:46497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.45.254:44487,127.18.45.253:38197,127.18.45.252:46495
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:49:38.437292 17849 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.45.253" port: 38197 } attempt: 1
I20260806 22:49:38.437920 17854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:38.437753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.253:38197 (local address 127.0.0.1:35334)
0806 22:49:38.437832 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:38.437876 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.253:38197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260806 22:49:38.438314 17854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:38.438205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.253:38197 (local address 127.0.0.1:35350)
0806 22:49:38.438254 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:38.438287 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.253:38197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260806 22:49:38.438412 17849 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.45.253:38197: Network error: Client connection negotiation failed: client connection to 127.18.45.253:38197: connect: Connection refused (error 111)
W20260806 22:49:38.498812 17859 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 22:49:38.499099 17859 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:38.499118 17859 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:38.499140 17859 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:38.500496 17859 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:49:38.500545 17859 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:38.500560 17859 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:38.500573 17859 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:49:38.500627 17859 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:49:38.502229 17859 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:46497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.18.45.254:44487,127.18.45.253:38197,127.18.45.252:46495
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.253.keytab
--principal=kudu/127.18.45.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.253:38197
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.45.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17859
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:49:38.502548 17859 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:38.502825 17859 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 22:49:38.506623 17849 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.45.253" port: 38197 } attempt: 2
I20260806 22:49:38.507339 17854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:38.507141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.253:38197 (local address 127.0.0.1:35360)
0806 22:49:38.507237 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:38.507285 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.253:38197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260806 22:49:38.507962 17854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:38.507864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.253:38197 (local address 127.0.0.1:35370)
0806 22:49:38.507908 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:38.507932 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.253:38197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260806 22:49:38.508076 17849 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.45.253:38197: Network error: Client connection negotiation failed: client connection to 127.18.45.253:38197: connect: Connection refused (error 111)
W20260806 22:49:38.508621 17865 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:38.508621 17868 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:38.508616 17864 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:38.509001 17859 server_base.cc:1034] running on GCE node
I20260806 22:49:38.509264 17859 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:38.509572 17859 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:38.510744 17859 hybrid_clock.cc:648] HybridClock initialized: now 1786056578510716 us; error 50 us; skew 500 ppm
Aug 06 22:49:38 dist-test-slave-h3ft krb5kdc[17741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056578, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:38.513950 17859 init.cc:377] Logged in from keytab as kudu/127.18.45.253@KRBTEST.COM (short username kudu)
I20260806 22:49:38.514482 17859 webserver.cc:460] Webserver started at http://127.18.45.253:38077/ using document root <none> and password file <none>
I20260806 22:49:38.514757 17859 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:38.514842 17859 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:38.515051 17859 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:49:38.516325 17859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/minicluster-data/master-1/data/instance:
uuid: "e23ccf2924134728a6e4518df5b3b7c8"
format_stamp: "Formatted at 2026-08-06 22:49:38 on dist-test-slave-h3ft"
server_key: "1f6cbe6dc70123cf2336ec5bf447114c"
server_key_iv: "cf0834cc6dd2452b46b9d5b8f025b474"
server_key_version: "encryptionkey@0"
I20260806 22:49:38.516758 17859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/minicluster-data/master-1/wal/instance:
uuid: "e23ccf2924134728a6e4518df5b3b7c8"
format_stamp: "Formatted at 2026-08-06 22:49:38 on dist-test-slave-h3ft"
server_key: "1f6cbe6dc70123cf2336ec5bf447114c"
server_key_iv: "cf0834cc6dd2452b46b9d5b8f025b474"
server_key_version: "encryptionkey@0"
I20260806 22:49:38.518302 17859 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:49:38.519145 17878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:38.519310 17859 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 22:49:38.519426 17859 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/minicluster-data/master-1/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/minicluster-data/master-1/wal
uuid: "e23ccf2924134728a6e4518df5b3b7c8"
format_stamp: "Formatted at 2026-08-06 22:49:38 on dist-test-slave-h3ft"
server_key: "1f6cbe6dc70123cf2336ec5bf447114c"
server_key_iv: "cf0834cc6dd2452b46b9d5b8f025b474"
server_key_version: "encryptionkey@0"
I20260806 22:49:38.519506 17859 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:38.532488 17859 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:38.534747 17859 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:38.534930 17859 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:38.540436 17859 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.253:38197
I20260806 22:49:38.540459 17939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.253:38197 every 8 connection(s)
I20260806 22:49:38.540961 17859 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/minicluster-data/master-1/data/info.pb
I20260806 22:49:38.542191 17940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:38.544175 17940 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.45.254" port: 44487 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:49:38.550678 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 17859
Aug 06 22:49:38 dist-test-slave-h3ft krb5kdc[17741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056578, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.253@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:49:38.550792 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/minicluster-data/master-1/wal/instance
I20260806 22:49:38.551074 18615 external_mini_cluster.cc:1428] Setting key 35469447ed2b09e5091cc671de6d3b66
I20260806 22:49:38.554678 17855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:38.545041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47226 (local address 127.18.45.254:44487)
0806 22:49:38.545101 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:38.545104 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:38.545370 (+ 266us) server_negotiation.cc:407] Connection header received
0806 22:49:38.545969 (+ 599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:38.545971 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:38.545986 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:38.546059 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:38.546820 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:38.547434 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:38.548518 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:38.548753 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:38.548833 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:38.551376 (+ 2543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:38.551378 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:38.551379 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:38.551390 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:38.553199 (+ 1809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:38.553645 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:38.553647 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:38.553647 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:38.553718 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:38.553985 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:38.553986 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:38.553986 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:38.554115 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:38.554189 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:38.554476 (+ 287us) server_negotiation.cc:299] Negotiation successful
0806 22:49:38.554573 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 22:49:38.554917 17942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:38.545010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:44487 (local address 127.0.0.1:47226)
0806 22:49:38.545225 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:38.545269 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:38.545835 (+ 566us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:38.546161 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:38.546165 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:38.546470 (+ 305us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:38.546712 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:38.546717 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:38.547531 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:38.547532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:38.548421 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:38.548423 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:38.548856 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:38.548857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:38.549005 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:38.549571 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:49:38.549580 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:38.551279 (+ 1699us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:38.553296 (+ 2017us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:38.553304 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:38.553306 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:38.553549 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:38.553791 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:38.553792 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:38.553792 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:38.553910 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:38.554189 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:38.554197 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:38.554343 (+ 146us) client_negotiation.cc:769] Sending connection context
0806 22:49:38.554453 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 22:49:38.554628 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":108,"threads_started":1}
I20260806 22:49:38.555702 17940 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.45.253" port: 38197 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:49:38 dist-test-slave-h3ft krb5kdc[17741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056578, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.253@KRBTEST.COM for kudu/127.18.45.253@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.45.252@KRBTEST.COM; defaulting to no policy
I20260806 22:49:38.563616 17942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:38.556055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.253:38197 (local address 127.0.0.1:57550)
0806 22:49:38.556112 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:38.556124 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:38.556171 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:38.556473 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:38.556474 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:38.556617 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:38.556666 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:38.556668 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:38.557637 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:38.557639 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:38.558138 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:38.558140 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:38.558647 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:38.558648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:38.558729 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:38.559198 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:49:38.559207 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:38.560435 (+ 1228us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:38.562245 (+ 1810us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:38.562246 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:38.562246 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:38.562521 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:38.562769 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:38.562770 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:38.562771 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:38.562841 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:38.563178 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:38.563179 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:38.563220 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 22:49:38.563294 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 22:49:38.563476 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260806 22:49:38.563629 17945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:38.556139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57550 (local address 127.18.45.253:38197)
0806 22:49:38.556315 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:38.556320 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:38.556350 (+ 30us) server_negotiation.cc:407] Connection header received
0806 22:49:38.556375 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:38.556377 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:38.556388 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:38.556473 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:38.556740 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:38.557534 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:38.558218 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:38.558563 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:38.558641 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:38.560531 (+ 1890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:38.560539 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:38.560540 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:38.560546 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:38.562143 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:38.562631 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:38.562642 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:38.562642 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:38.562705 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:38.562903 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:38.562904 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:38.562905 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:38.563111 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:38.563204 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:38.563335 (+ 131us) server_negotiation.cc:299] Negotiation successful
0806 22:49:38.563502 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":98,"server-negotiator.queue_time_us":144,"thread_start_us":114,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.252@KRBTEST.COM" created.
I20260806 22:49:38.564438 17940 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.45.252" port: 46495 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:49:38.564838 17942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:38.564706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.252:46495 (local address 127.0.0.1:34122)
0806 22:49:38.564760 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:38.564811 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.252:46495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260806 22:49:38.564987 17879 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.45.252:46495: connect: Connection refused (error 111)
I20260806 22:49:38.565486 17942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:38.565383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.252:46495 (local address 127.0.0.1:34134)
0806 22:49:38.565430 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:38.565457 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.252:46495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260806 22:49:38.565611 17940 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.45.252:46495: Network error: Client connection negotiation failed: client connection to 127.18.45.252:46495: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.252.keytab.
Entry for principal kudu/127.18.45.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.252.keytab.
I20260806 22:49:38.581851 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.252:46495
--webserver_interface=127.18.45.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.252.keytab
--principal=kudu/127.18.45.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:46497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.45.254:44487,127.18.45.253:38197,127.18.45.252:46495
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:49:38.599313 17849 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.45.253" port: 38197 } attempt: 3
Aug 06 22:49:38 dist-test-slave-h3ft krb5kdc[17741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056578, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.254@KRBTEST.COM for kudu/127.18.45.253@KRBTEST.COM
I20260806 22:49:38.607492 17854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:38.599844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.253:38197 (local address 127.0.0.1:57560)
0806 22:49:38.599908 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:38.599923 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:38.599987 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:38.600173 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:38.600174 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:38.600384 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:38.600477 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:38.600480 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:38.601237 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:38.601239 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:38.601796 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:38.601798 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:38.602253 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:38.602255 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:38.602352 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:38.602927 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:49:38.602932 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:38.604410 (+ 1478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:38.606239 (+ 1829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:38.606240 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:38.606241 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:38.606433 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:38.606686 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:38.606687 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:38.606688 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:38.606758 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:38.607081 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:38.607082 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:38.607140 (+ 58us) client_negotiation.cc:769] Sending connection context
0806 22:49:38.607239 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 22:49:38.607370 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260806 22:49:38.607491 17945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:38.599903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57560 (local address 127.18.45.253:38197)
0806 22:49:38.600021 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:38.600024 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:38.600037 (+ 13us) server_negotiation.cc:407] Connection header received
0806 22:49:38.600076 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:38.600077 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:38.600095 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:38.600162 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:38.600577 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:38.601136 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:38.601889 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:38.602171 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:38.602255 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:38.604519 (+ 2264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:38.604521 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:38.604522 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:38.604531 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:38.606133 (+ 1602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:38.606534 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:38.606536 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:38.606536 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:38.606612 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:38.606821 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:38.606822 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:38.606823 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:38.606966 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:38.607088 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:38.607269 (+ 181us) server_negotiation.cc:299] Negotiation successful
0806 22:49:38.607370 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260806 22:49:38.607981 17849 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.45.252" port: 46495 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:49:38.608455 17854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:38.608335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.252:46495 (local address 127.0.0.1:34144)
0806 22:49:38.608389 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:38.608425 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.252:46495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260806 22:49:38.608784 17854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:38.608696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.252:46495 (local address 127.0.0.1:34158)
0806 22:49:38.608738 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:38.608763 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.252:46495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260806 22:49:38.608877 17849 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.45.252:46495: Network error: Client connection negotiation failed: client connection to 127.18.45.252:46495: connect: Connection refused (error 111)
I20260806 22:49:38.614850 17940 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.45.252" port: 46495 } attempt: 1
I20260806 22:49:38.615411 17942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:38.615255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.252:46495 (local address 127.0.0.1:34168)
0806 22:49:38.615318 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:38.615367 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.252:46495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 22:49:38.615808 17942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:38.615709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.252:46495 (local address 127.0.0.1:34172)
0806 22:49:38.615756 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:38.615787 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.252:46495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260806 22:49:38.615931 17940 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.45.252:46495: Network error: Client connection negotiation failed: client connection to 127.18.45.252:46495: connect: Connection refused (error 111)
I20260806 22:49:38.640026 17849 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.45.252" port: 46495 } attempt: 1
I20260806 22:49:38.640614 17854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:38.640452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.252:46495 (local address 127.0.0.1:34180)
0806 22:49:38.640529 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:38.640570 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.252:46495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260806 22:49:38.641072 17854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:38.640973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.252:46495 (local address 127.0.0.1:34190)
0806 22:49:38.641019 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:38.641048 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.252:46495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260806 22:49:38.641199 17849 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.45.252:46495: Network error: Client connection negotiation failed: client connection to 127.18.45.252:46495: connect: Connection refused (error 111)
W20260806 22:49:38.679692 17948 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 22:49:38.679970 17948 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:38.679989 17948 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:38.680013 17948 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:38.681350 17948 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:49:38.681401 17948 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:38.681416 17948 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:38.681428 17948 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:49:38.681440 17948 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:49:38.683066 17948 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:46497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.18.45.254:44487,127.18.45.253:38197,127.18.45.252:46495
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.252.keytab
--principal=kudu/127.18.45.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.252:46495
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.45.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17948
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:49:38.683328 17948 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:38.683559 17948 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 22:49:38.684124 17940 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.45.252" port: 46495 } attempt: 2
I20260806 22:49:38.684801 17942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:38.684606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.252:46495 (local address 127.0.0.1:34198)
0806 22:49:38.684694 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:38.684753 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.252:46495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260806 22:49:38.685252 17942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:38.685132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.252:46495 (local address 127.0.0.1:34208)
0806 22:49:38.685180 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:38.685213 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.252:46495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260806 22:49:38.685371 17940 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.45.252:46495: Network error: Client connection negotiation failed: client connection to 127.18.45.252:46495: connect: Connection refused (error 111)
W20260806 22:49:38.689606 17955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:38.689606 17954 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:38.689651 17958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:38.690008 17948 server_base.cc:1034] running on GCE node
I20260806 22:49:38.690354 17948 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:38.690730 17948 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:38.691908 17948 hybrid_clock.cc:648] HybridClock initialized: now 1786056578691891 us; error 44 us; skew 500 ppm
Aug 06 22:49:38 dist-test-slave-h3ft krb5kdc[17741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056578, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:38.695113 17948 init.cc:377] Logged in from keytab as kudu/127.18.45.252@KRBTEST.COM (short username kudu)
I20260806 22:49:38.695675 17948 webserver.cc:460] Webserver started at http://127.18.45.252:42363/ using document root <none> and password file <none>
I20260806 22:49:38.695952 17948 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:38.696024 17948 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:38.696198 17948 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:49:38.697504 17948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/minicluster-data/master-2/data/instance:
uuid: "0e2a9a44b84c4912b44c3928f12b9b7c"
format_stamp: "Formatted at 2026-08-06 22:49:38 on dist-test-slave-h3ft"
server_key: "d96b915cbd0b61f4f7f998b60797fb64"
server_key_iv: "14da5a0d11f2fd0c03ebd325567d2b31"
server_key_version: "encryptionkey@0"
I20260806 22:49:38.697937 17948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/minicluster-data/master-2/wal/instance:
uuid: "0e2a9a44b84c4912b44c3928f12b9b7c"
format_stamp: "Formatted at 2026-08-06 22:49:38 on dist-test-slave-h3ft"
server_key: "d96b915cbd0b61f4f7f998b60797fb64"
server_key_iv: "14da5a0d11f2fd0c03ebd325567d2b31"
server_key_version: "encryptionkey@0"
I20260806 22:49:38.699697 17948 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260806 22:49:38.700555 17968 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:38.700742 17948 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:49:38.700860 17948 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/minicluster-data/master-2/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/minicluster-data/master-2/wal
uuid: "0e2a9a44b84c4912b44c3928f12b9b7c"
format_stamp: "Formatted at 2026-08-06 22:49:38 on dist-test-slave-h3ft"
server_key: "d96b915cbd0b61f4f7f998b60797fb64"
server_key_iv: "14da5a0d11f2fd0c03ebd325567d2b31"
server_key_version: "encryptionkey@0"
I20260806 22:49:38.700941 17948 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:38.716406 17849 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.45.252" port: 46495 } attempt: 2
I20260806 22:49:38.717058 17854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:38.716893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.252:46495 (local address 127.0.0.1:34224)
0806 22:49:38.716965 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:38.717014 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.252:46495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260806 22:49:38.717479 17854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:38.717374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.252:46495 (local address 127.0.0.1:34234)
0806 22:49:38.717424 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:38.717450 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.252:46495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260806 22:49:38.717566 17849 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.45.252:46495: Network error: Client connection negotiation failed: client connection to 127.18.45.252:46495: connect: Connection refused (error 111)
I20260806 22:49:38.733803 17948 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:38.736809 17948 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:38.737010 17948 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:38.742125 17948 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.252:46495
I20260806 22:49:38.742197 18042 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.252:46495 every 8 connection(s)
I20260806 22:49:38.742624 17948 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/minicluster-data/master-2/data/info.pb
I20260806 22:49:38.744174 18043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:38.746368 18043 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.45.254" port: 44487 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:49:38.749626 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 17948
I20260806 22:49:38.749754 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786056275803658-18615-0/minicluster-data/master-2/wal/instance
I20260806 22:49:38.749953 18615 external_mini_cluster.cc:1428] Setting key f341bb7697214bdeddd3b29c2dbdd14e
Aug 06 22:49:38 dist-test-slave-h3ft krb5kdc[17741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056578, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.252@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
Aug 06 22:49:38 dist-test-slave-h3ft krb5kdc[17741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056578, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.45.252@KRBTEST.COM
I20260806 22:49:38.757404 17855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:38.747286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47242 (local address 127.18.45.254:44487)
0806 22:49:38.747401 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:38.747403 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:38.747615 (+ 212us) server_negotiation.cc:407] Connection header received
0806 22:49:38.748172 (+ 557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:38.748173 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:38.748185 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:38.748245 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:38.748987 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:38.749523 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:38.750988 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:38.751233 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:38.751321 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:38.754040 (+ 2719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:38.754041 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:38.754042 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:38.754049 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:38.755701 (+ 1652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:38.756245 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:38.756246 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:38.756247 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:38.756316 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:38.756629 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:38.756630 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:38.756630 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:38.756768 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:38.756827 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:38.757187 (+ 360us) server_negotiation.cc:299] Negotiation successful
0806 22:49:38.757291 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 22:49:38.757604 18046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:38.747217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:44487 (local address 127.0.0.1:47242)
0806 22:49:38.747456 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:38.747500 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:38.748072 (+ 572us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:38.748315 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:38.748319 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:38.748641 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:38.748880 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:38.748884 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:38.749660 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:38.749661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:38.750883 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:38.750887 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:38.751356 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:38.751357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:38.751506 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:38.752150 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:49:38.752159 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:38.753941 (+ 1782us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:38.755826 (+ 1885us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:38.755835 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:38.755839 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:38.756169 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:38.756394 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:38.756395 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:38.756395 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:38.756551 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:38.756835 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:38.756844 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:38.757040 (+ 196us) client_negotiation.cc:769] Sending connection context
0806 22:49:38.757139 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 22:49:38.757281 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":142,"threads_started":1}
I20260806 22:49:38.758389 18043 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.45.253" port: 38197 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:49:38.760061 18048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:38.750428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34240 (local address 127.18.45.252:46495)
0806 22:49:38.750648 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:38.750653 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:38.750746 (+ 93us) server_negotiation.cc:407] Connection header received
0806 22:49:38.750818 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:38.750821 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:38.750844 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:38.750942 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:38.751670 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:38.752383 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:38.753424 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:38.753810 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:38.753882 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:38.756437 (+ 2555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:38.756447 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:38.756447 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:38.756454 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:38.758665 (+ 2211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:38.759130 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:38.759134 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:38.759134 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:38.759209 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:38.759440 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:38.759441 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:38.759442 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:38.759632 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:38.759798 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:38.759862 (+ 64us) server_negotiation.cc:299] Negotiation successful
0806 22:49:38.759944 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":115,"threads_started":1}
Aug 06 22:49:38 dist-test-slave-h3ft krb5kdc[17741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056578, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.252@KRBTEST.COM for kudu/127.18.45.253@KRBTEST.COM
Aug 06 22:49:38 dist-test-slave-h3ft krb5kdc[17741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056578, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:49:38.766835 17945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:38.758880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57572 (local address 127.18.45.253:38197)
0806 22:49:38.758946 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:38.758949 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:38.758959 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:49:38.758994 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:38.758995 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:38.759042 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:38.759104 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:38.759390 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:38.760134 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:38.761069 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:38.761314 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:38.761388 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:38.763620 (+ 2232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:38.763622 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:38.763622 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:38.763631 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:38.765590 (+ 1959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:38.765976 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:38.765979 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:38.765980 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:38.766057 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:38.766309 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:38.766310 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:38.766310 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:38.766429 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:38.766483 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:38.766647 (+ 164us) server_negotiation.cc:299] Negotiation successful
0806 22:49:38.766749 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 22:49:38.766845 18046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:38.758760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.253:38197 (local address 127.0.0.1:57572)
0806 22:49:38.758802 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:38.758814 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:38.758847 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:38.759109 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:38.759110 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:38.759251 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:38.759306 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:38.759308 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:38.760212 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:38.760213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:38.760977 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:38.760980 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:38.761395 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:38.761396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:38.761463 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:38.761907 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:49:38.761912 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:38.763514 (+ 1602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:38.765690 (+ 2176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:38.765692 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:38.765692 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:38.765892 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:38.766144 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:38.766146 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:38.766146 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:38.766231 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:38.766494 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:38.766495 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:38.766548 (+ 53us) client_negotiation.cc:769] Sending connection context
0806 22:49:38.766628 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 22:49:38.766738 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260806 22:49:38.767360 18043 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.45.252" port: 46495 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:49:38.770183 17855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:38.761037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47248 (local address 127.18.45.254:44487)
0806 22:49:38.761463 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:38.761465 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:38.761474 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:49:38.761496 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:38.761497 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:38.761508 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:38.761563 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:38.762034 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:38.762692 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:38.763949 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:38.764260 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:38.764316 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:38.767318 (+ 3002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:38.767320 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:38.767320 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:38.767328 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:38.768901 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:38.769217 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:38.769218 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:38.769218 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:38.769285 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:38.769434 (+ 149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:38.769434 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:38.769435 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:38.769567 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:38.769751 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:38.769983 (+ 232us) server_negotiation.cc:299] Negotiation successful
0806 22:49:38.770053 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401}
Aug 06 22:49:38 dist-test-slave-h3ft krb5kdc[17741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056578, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.252@KRBTEST.COM for kudu/127.18.45.252@KRBTEST.COM
I20260806 22:49:38.774488 18046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:38.767731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.252:46495 (local address 127.0.0.1:34250)
0806 22:49:38.767804 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:38.767814 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:38.767877 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:38.767993 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:38.767994 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:38.768156 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:38.768197 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:38.768199 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:38.768837 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:38.768838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:38.769554 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:38.769556 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:38.769919 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:38.769920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:38.769977 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:38.770441 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:49:38.770447 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:38.771821 (+ 1374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:38.773469 (+ 1648us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:38.773470 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:38.773470 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:38.773639 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:38.773856 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:38.773856 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:38.773857 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:38.773914 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:38.774153 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:38.774154 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:38.774193 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 22:49:38.774258 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 22:49:38.774370 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260806 22:49:38.774499 18048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:38.767729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34250 (local address 127.18.45.252:46495)
0806 22:49:38.767778 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:38.767780 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:38.767853 (+ 73us) server_negotiation.cc:407] Connection header received
0806 22:49:38.767924 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:38.767924 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:38.767935 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:38.767990 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:38.768266 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:38.768756 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:38.769627 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:38.769851 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:38.769901 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:38.771926 (+ 2025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:38.771928 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:38.771928 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:38.771935 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:38.773369 (+ 1434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:38.773719 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:38.773720 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:38.773721 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:38.773775 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:38.773972 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:38.773973 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:38.773974 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:38.774088 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:38.774163 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:38.774284 (+ 121us) server_negotiation.cc:299] Negotiation successful
0806 22:49:38.774418 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":67,"server-negotiator.queue_time_us":25}
Aug 06 22:49:38 dist-test-slave-h3ft krb5kdc[17741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056578, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.45.253@KRBTEST.COM
I20260806 22:49:38.776594 17940 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.45.252" port: 46495 } attempt: 3
I20260806 22:49:38.776664 18043 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e2a9a44b84c4912b44c3928f12b9b7c: Bootstrap starting.
I20260806 22:49:38.777980 18043 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0e2a9a44b84c4912b44c3928f12b9b7c: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:38.778409 18043 log.cc:824] T 00000000000000000000000000000000 P 0e2a9a44b84c4912b44c3928f12b9b7c: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:38.779110 17945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:38.771147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57584 (local address 127.18.45.253:38197)
0806 22:49:38.771194 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:38.771195 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:38.771202 (+ 7us) server_negotiation.cc:407] Connection header received
0806 22:49:38.771219 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:38.771220 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:38.771229 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:38.771285 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:38.771806 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:38.772285 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:38.772963 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:38.773179 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:38.773248 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:38.775638 (+ 2390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:38.775640 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:38.775640 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:38.775649 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:38.777734 (+ 2085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:38.778169 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:38.778170 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:38.778170 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:38.778225 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:38.778443 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:38.778444 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:38.778445 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:38.778611 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:38.778843 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:38.778886 (+ 43us) server_negotiation.cc:299] Negotiation successful
0806 22:49:38.778978 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 22:49:38.780222 18043 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e2a9a44b84c4912b44c3928f12b9b7c: No bootstrap required, opened a new log
I20260806 22:49:38.781103 18615 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
Aug 06 22:49:38 dist-test-slave-h3ft krb5kdc[17741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056578, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.253@KRBTEST.COM for kudu/127.18.45.252@KRBTEST.COM
I20260806 22:49:38.782887 18043 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0e2a9a44b84c4912b44c3928f12b9b7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64ab4b7cbb194edf8487645652a9954e" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 44487 } } peers { permanent_uuid: "e23ccf2924134728a6e4518df5b3b7c8" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 38197 } } peers { permanent_uuid: "0e2a9a44b84c4912b44c3928f12b9b7c" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 46495 } }
I20260806 22:49:38.783115 18043 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0e2a9a44b84c4912b44c3928f12b9b7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e2a9a44b84c4912b44c3928f12b9b7c, State: Initialized, Role: FOLLOWER
I20260806 22:49:38.783233 18043 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e2a9a44b84c4912b44c3928f12b9b7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64ab4b7cbb194edf8487645652a9954e" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 44487 } } peers { permanent_uuid: "e23ccf2924134728a6e4518df5b3b7c8" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 38197 } } peers { permanent_uuid: "0e2a9a44b84c4912b44c3928f12b9b7c" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 46495 } }
I20260806 22:49:38.783691 18043 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0e2a9a44b84c4912b44c3928f12b9b7c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:49:38.783643 18051 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e2a9a44b84c4912b44c3928f12b9b7c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64ab4b7cbb194edf8487645652a9954e" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 44487 } } peers { permanent_uuid: "e23ccf2924134728a6e4518df5b3b7c8" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 38197 } } peers { permanent_uuid: "0e2a9a44b84c4912b44c3928f12b9b7c" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 46495 } } }
I20260806 22:49:38.782737 17807 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "64ab4b7cbb194edf8487645652a9954e"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47248
I20260806 22:49:38.783944 18051 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e2a9a44b84c4912b44c3928f12b9b7c [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:49:38.785043 18048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:38.776975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34256 (local address 127.18.45.252:46495)
0806 22:49:38.777033 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:38.777035 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:38.777061 (+ 26us) server_negotiation.cc:407] Connection header received
0806 22:49:38.777084 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:38.777085 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:38.777102 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:38.777168 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:38.777503 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:38.778194 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:38.779202 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:38.779487 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:38.779599 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:38.781884 (+ 2285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:38.781886 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:38.781886 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:38.781894 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:38.783715 (+ 1821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:38.784155 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:38.784156 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:38.784156 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:38.784231 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:38.784432 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:38.784433 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:38.784434 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:38.784616 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:38.784686 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:38.784857 (+ 171us) server_negotiation.cc:299] Negotiation successful
0806 22:49:38.784946 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 22:49:38.785050 17942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:38.776916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.252:46495 (local address 127.0.0.1:34256)
0806 22:49:38.776968 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:38.776980 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:38.777029 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:38.777193 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:38.777194 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:38.777355 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:38.777414 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:38.777417 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:38.778307 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:38.778309 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:38.779106 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:38.779109 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:38.779626 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:38.779627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:38.779700 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:38.780169 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:49:38.780174 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:38.781779 (+ 1605us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:38.783827 (+ 2048us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:38.783829 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:38.783829 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:38.784064 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:38.784308 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:38.784310 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:38.784310 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:38.784381 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:38.784677 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:38.784678 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:38.784746 (+ 68us) client_negotiation.cc:769] Sending connection context
0806 22:49:38.784830 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 22:49:38.784949 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
W20260806 22:49:38.785952 18062 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0e2a9a44b84c4912b44c3928f12b9b7c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:49:38.786016 18062 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:49:38.787997 17940 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e23ccf2924134728a6e4518df5b3b7c8: Bootstrap starting.
I20260806 22:49:38.788937 17940 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e23ccf2924134728a6e4518df5b3b7c8: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:38.786324 17948 master_runner.cc:186] Error getting master registration for 127.18.45.254:44487: OK, instance_id { permanent_uuid: "64ab4b7cbb194edf8487645652a9954e" instance_seqno: 1786056578367911 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 22:49:38.789327 17948 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 22:49:38.789309 17940 log.cc:824] T 00000000000000000000000000000000 P e23ccf2924134728a6e4518df5b3b7c8: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:38.790314 17940 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e23ccf2924134728a6e4518df5b3b7c8: No bootstrap required, opened a new log
I20260806 22:49:38.792068 17940 raft_consensus.cc:348] T 00000000000000000000000000000000 P e23ccf2924134728a6e4518df5b3b7c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64ab4b7cbb194edf8487645652a9954e" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 44487 } } peers { permanent_uuid: "e23ccf2924134728a6e4518df5b3b7c8" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 38197 } } peers { permanent_uuid: "0e2a9a44b84c4912b44c3928f12b9b7c" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 46495 } }
I20260806 22:49:38.792213 17940 raft_consensus.cc:729] T 00000000000000000000000000000000 P e23ccf2924134728a6e4518df5b3b7c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e23ccf2924134728a6e4518df5b3b7c8, State: Initialized, Role: FOLLOWER
I20260806 22:49:38.792315 17940 consensus_queue.cc:260] T 00000000000000000000000000000000 P e23ccf2924134728a6e4518df5b3b7c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64ab4b7cbb194edf8487645652a9954e" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 44487 } } peers { permanent_uuid: "e23ccf2924134728a6e4518df5b3b7c8" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 38197 } } peers { permanent_uuid: "0e2a9a44b84c4912b44c3928f12b9b7c" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 46495 } }
I20260806 22:49:38.792670 18067 sys_catalog.cc:455] T 00000000000000000000000000000000 P e23ccf2924134728a6e4518df5b3b7c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64ab4b7cbb194edf8487645652a9954e" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 44487 } } peers { permanent_uuid: "e23ccf2924134728a6e4518df5b3b7c8" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 38197 } } peers { permanent_uuid: "0e2a9a44b84c4912b44c3928f12b9b7c" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 46495 } } }
I20260806 22:49:38.792754 17940 sys_catalog.cc:564] T 00000000000000000000000000000000 P e23ccf2924134728a6e4518df5b3b7c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:49:38.792786 18067 sys_catalog.cc:458] T 00000000000000000000000000000000 P e23ccf2924134728a6e4518df5b3b7c8 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 22:49:38.794917 18080 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e23ccf2924134728a6e4518df5b3b7c8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:49:38.794987 18080 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:49:38.795195 17859 master_runner.cc:186] Error getting master registration for 127.18.45.254:44487: OK, instance_id { permanent_uuid: "64ab4b7cbb194edf8487645652a9954e" instance_seqno: 1786056578367911 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 22:49:38.798453 17859 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 22:49:38.816725 17849 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.45.252" port: 46495 } attempt: 3
Aug 06 22:49:38 dist-test-slave-h3ft krb5kdc[17741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056578, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.254@KRBTEST.COM for kudu/127.18.45.252@KRBTEST.COM
I20260806 22:49:38.826048 18048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:38.817552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34272 (local address 127.18.45.252:46495)
0806 22:49:38.817619 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:38.817621 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:38.817631 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:49:38.817667 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:38.817668 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:38.817684 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:38.817755 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:38.818155 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:38.818968 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:38.820048 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:38.820264 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:38.820355 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:38.823189 (+ 2834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:38.823191 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:38.823191 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:38.823200 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:38.824792 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:38.825187 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:38.825188 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:38.825189 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:38.825251 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:38.825466 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:38.825467 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:38.825468 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:38.825612 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:38.825672 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:38.825842 (+ 170us) server_negotiation.cc:299] Negotiation successful
0806 22:49:38.825932 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 22:49:38.826048 17854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:38.817279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.252:46495 (local address 127.0.0.1:34272)
0806 22:49:38.817370 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:38.817384 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:38.817443 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:38.817765 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:38.817766 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:38.817997 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:38.818058 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:38.818060 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:38.819087 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:38.819089 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:38.819972 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:38.819975 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:38.820354 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:38.820355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:38.820420 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:38.821257 (+ 837us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:49:38.821264 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:38.823098 (+ 1834us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:38.824888 (+ 1790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:38.824890 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:38.824890 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:38.825125 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:38.825323 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:38.825324 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:38.825325 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:38.825391 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:38.825678 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:38.825679 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:38.825729 (+ 50us) client_negotiation.cc:769] Sending connection context
0806 22:49:38.825812 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 22:49:38.825931 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260806 22:49:38.828593 17849 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64ab4b7cbb194edf8487645652a9954e: Bootstrap starting.
I20260806 22:49:38.829555 17849 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 64ab4b7cbb194edf8487645652a9954e: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:38.829948 17849 log.cc:824] T 00000000000000000000000000000000 P 64ab4b7cbb194edf8487645652a9954e: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:38.830935 17849 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64ab4b7cbb194edf8487645652a9954e: No bootstrap required, opened a new log
I20260806 22:49:38.832597 17849 raft_consensus.cc:348] T 00000000000000000000000000000000 P 64ab4b7cbb194edf8487645652a9954e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64ab4b7cbb194edf8487645652a9954e" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 44487 } } peers { permanent_uuid: "e23ccf2924134728a6e4518df5b3b7c8" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 38197 } } peers { permanent_uuid: "0e2a9a44b84c4912b44c3928f12b9b7c" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 46495 } }
I20260806 22:49:38.832767 17849 raft_consensus.cc:729] T 00000000000000000000000000000000 P 64ab4b7cbb194edf8487645652a9954e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64ab4b7cbb194edf8487645652a9954e, State: Initialized, Role: FOLLOWER
I20260806 22:49:38.832866 17849 consensus_queue.cc:260] T 00000000000000000000000000000000 P 64ab4b7cbb194edf8487645652a9954e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64ab4b7cbb194edf8487645652a9954e" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 44487 } } peers { permanent_uuid: "e23ccf2924134728a6e4518df5b3b7c8" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 38197 } } peers { permanent_uuid: "0e2a9a44b84c4912b44c3928f12b9b7c" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 46495 } }
I20260806 22:49:38.833230 17849 sys_catalog.cc:564] T 00000000000000000000000000000000 P 64ab4b7cbb194edf8487645652a9954e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:49:38.833248 17807 raft_consensus.cc:482] T 00000000000000000000000000000000 P 64ab4b7cbb194edf8487645652a9954e [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 22:49:38.833298 17807 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 64ab4b7cbb194edf8487645652a9954e [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:38.834323 17807 raft_consensus.cc:504] T 00000000000000000000000000000000 P 64ab4b7cbb194edf8487645652a9954e [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64ab4b7cbb194edf8487645652a9954e" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 44487 } } peers { permanent_uuid: "e23ccf2924134728a6e4518df5b3b7c8" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 38197 } } peers { permanent_uuid: "0e2a9a44b84c4912b44c3928f12b9b7c" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 46495 } }
I20260806 22:49:38.834679 17807 leader_election.cc:288] T 00000000000000000000000000000000 P 64ab4b7cbb194edf8487645652a9954e [CANDIDATE]: Term 1 election: Requested vote from peers e23ccf2924134728a6e4518df5b3b7c8 (127.18.45.253:38197), 0e2a9a44b84c4912b44c3928f12b9b7c (127.18.45.252:46495)
I20260806 22:49:38.833173 18084 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64ab4b7cbb194edf8487645652a9954e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64ab4b7cbb194edf8487645652a9954e" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 44487 } } peers { permanent_uuid: "e23ccf2924134728a6e4518df5b3b7c8" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 38197 } } peers { permanent_uuid: "0e2a9a44b84c4912b44c3928f12b9b7c" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 46495 } } }
I20260806 22:49:38.835124 17910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "64ab4b7cbb194edf8487645652a9954e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "e23ccf2924134728a6e4518df5b3b7c8"
I20260806 22:49:38.835304 17910 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e23ccf2924134728a6e4518df5b3b7c8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:38.835251 18004 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "64ab4b7cbb194edf8487645652a9954e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "0e2a9a44b84c4912b44c3928f12b9b7c"
I20260806 22:49:38.835417 18004 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0e2a9a44b84c4912b44c3928f12b9b7c [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:38.836246 18084 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64ab4b7cbb194edf8487645652a9954e [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:49:38.836467 17910 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e23ccf2924134728a6e4518df5b3b7c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64ab4b7cbb194edf8487645652a9954e in term 1.
I20260806 22:49:38.836647 18004 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0e2a9a44b84c4912b44c3928f12b9b7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64ab4b7cbb194edf8487645652a9954e in term 1.
I20260806 22:49:38.836845 17776 leader_election.cc:302] T 00000000000000000000000000000000 P 64ab4b7cbb194edf8487645652a9954e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64ab4b7cbb194edf8487645652a9954e, e23ccf2924134728a6e4518df5b3b7c8; no voters:
I20260806 22:49:38.837064 18084 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 64ab4b7cbb194edf8487645652a9954e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:49:38.837802 18084 raft_consensus.cc:686] T 00000000000000000000000000000000 P 64ab4b7cbb194edf8487645652a9954e [term 1 LEADER]: Becoming Leader. State: Replica: 64ab4b7cbb194edf8487645652a9954e, State: Running, Role: LEADER
I20260806 22:49:38.837971 18084 consensus_queue.cc:237] T 00000000000000000000000000000000 P 64ab4b7cbb194edf8487645652a9954e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64ab4b7cbb194edf8487645652a9954e" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 44487 } } peers { permanent_uuid: "e23ccf2924134728a6e4518df5b3b7c8" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 38197 } } peers { permanent_uuid: "0e2a9a44b84c4912b44c3928f12b9b7c" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 46495 } }
I20260806 22:49:38.839184 18106 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64ab4b7cbb194edf8487645652a9954e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64ab4b7cbb194edf8487645652a9954e. Latest consensus state: current_term: 1 leader_uuid: "64ab4b7cbb194edf8487645652a9954e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64ab4b7cbb194edf8487645652a9954e" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 44487 } } peers { permanent_uuid: "e23ccf2924134728a6e4518df5b3b7c8" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 38197 } } peers { permanent_uuid: "0e2a9a44b84c4912b44c3928f12b9b7c" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 46495 } } }
I20260806 22:49:38.839282 18106 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64ab4b7cbb194edf8487645652a9954e [sys.catalog]: This master's current role is: LEADER
W20260806 22:49:38.839314 18105 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 64ab4b7cbb194edf8487645652a9954e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:49:38.839375 18105 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:49:38.847249 18110 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:49:38.847572 18110 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:49:38.849623 18004 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0e2a9a44b84c4912b44c3928f12b9b7c [term 1 FOLLOWER]: Refusing update from remote peer 64ab4b7cbb194edf8487645652a9954e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:49:38.850402 18106 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 64ab4b7cbb194edf8487645652a9954e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e2a9a44b84c4912b44c3928f12b9b7c" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 46495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:49:38.853214 18051 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e2a9a44b84c4912b44c3928f12b9b7c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64ab4b7cbb194edf8487645652a9954e. Latest consensus state: current_term: 1 leader_uuid: "64ab4b7cbb194edf8487645652a9954e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64ab4b7cbb194edf8487645652a9954e" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 44487 } } peers { permanent_uuid: "e23ccf2924134728a6e4518df5b3b7c8" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 38197 } } peers { permanent_uuid: "0e2a9a44b84c4912b44c3928f12b9b7c" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 46495 } } }
I20260806 22:49:38.853350 18051 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e2a9a44b84c4912b44c3928f12b9b7c [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:49:38.854053 17910 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e23ccf2924134728a6e4518df5b3b7c8 [term 1 FOLLOWER]: Refusing update from remote peer 64ab4b7cbb194edf8487645652a9954e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:49:38.854426 18106 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 64ab4b7cbb194edf8487645652a9954e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e23ccf2924134728a6e4518df5b3b7c8" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 38197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:49:38.856057 18067 sys_catalog.cc:455] T 00000000000000000000000000000000 P e23ccf2924134728a6e4518df5b3b7c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64ab4b7cbb194edf8487645652a9954e. Latest consensus state: current_term: 1 leader_uuid: "64ab4b7cbb194edf8487645652a9954e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64ab4b7cbb194edf8487645652a9954e" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 44487 } } peers { permanent_uuid: "e23ccf2924134728a6e4518df5b3b7c8" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 38197 } } peers { permanent_uuid: "0e2a9a44b84c4912b44c3928f12b9b7c" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 46495 } } }
I20260806 22:49:38.856170 18067 sys_catalog.cc:458] T 00000000000000000000000000000000 P e23ccf2924134728a6e4518df5b3b7c8 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:49:38.860425 18084 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64ab4b7cbb194edf8487645652a9954e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "64ab4b7cbb194edf8487645652a9954e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64ab4b7cbb194edf8487645652a9954e" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 44487 } } peers { permanent_uuid: "e23ccf2924134728a6e4518df5b3b7c8" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 38197 } } peers { permanent_uuid: "0e2a9a44b84c4912b44c3928f12b9b7c" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 46495 } } }
I20260806 22:49:38.860553 18084 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64ab4b7cbb194edf8487645652a9954e [sys.catalog]: This master's current role is: LEADER
I20260806 22:49:38.861218 18051 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e2a9a44b84c4912b44c3928f12b9b7c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "64ab4b7cbb194edf8487645652a9954e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64ab4b7cbb194edf8487645652a9954e" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 44487 } } peers { permanent_uuid: "e23ccf2924134728a6e4518df5b3b7c8" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 38197 } } peers { permanent_uuid: "0e2a9a44b84c4912b44c3928f12b9b7c" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 46495 } } }
I20260806 22:49:38.861346 18051 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e2a9a44b84c4912b44c3928f12b9b7c [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:49:38.861780 18114 mvcc.cc:205] Tried to move back new op lower bound from 7315687746964635648 to 7315687746921213952. Current Snapshot: MvccSnapshot[applied={T|T < 7315687746964635648}]
I20260806 22:49:38.862898 18106 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64ab4b7cbb194edf8487645652a9954e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "64ab4b7cbb194edf8487645652a9954e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64ab4b7cbb194edf8487645652a9954e" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 44487 } } peers { permanent_uuid: "e23ccf2924134728a6e4518df5b3b7c8" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 38197 } } peers { permanent_uuid: "0e2a9a44b84c4912b44c3928f12b9b7c" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 46495 } } }
I20260806 22:49:38.862989 18106 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64ab4b7cbb194edf8487645652a9954e [sys.catalog]: This master's current role is: LEADER
I20260806 22:49:38.863662 18067 sys_catalog.cc:455] T 00000000000000000000000000000000 P e23ccf2924134728a6e4518df5b3b7c8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "64ab4b7cbb194edf8487645652a9954e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64ab4b7cbb194edf8487645652a9954e" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 44487 } } peers { permanent_uuid: "e23ccf2924134728a6e4518df5b3b7c8" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 38197 } } peers { permanent_uuid: "0e2a9a44b84c4912b44c3928f12b9b7c" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 46495 } } }
I20260806 22:49:38.863766 18067 sys_catalog.cc:458] T 00000000000000000000000000000000 P e23ccf2924134728a6e4518df5b3b7c8 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:49:38.864079 18117 mvcc.cc:205] Tried to move back new op lower bound from 7315687746964635648 to 7315687746921213952. Current Snapshot: MvccSnapshot[applied={T|T < 7315687746964635648}]
I20260806 22:49:38.868716 18110 catalog_manager.cc:1347] Generated new cluster ID: 8cfc8e347aff4c7398370d9da16955e6
I20260806 22:49:38.868781 18110 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:49:38.872696 18048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:38.843338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34284 (local address 127.18.45.252:46495)
0806 22:49:38.843432 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:38.843435 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:38.844354 (+ 919us) server_negotiation.cc:407] Connection header received
0806 22:49:38.844492 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:38.844493 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:38.844506 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:38.844539 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:38.855763 (+ 11224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:38.856579 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:38.861493 (+ 4914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:38.861797 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:38.861847 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:38.868774 (+ 6927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:38.868776 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:38.868776 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:38.868787 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:38.870563 (+ 1776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:38.871726 (+ 1163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:38.871728 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:38.871729 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:38.871796 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:38.871994 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:38.871995 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:38.871995 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:38.872149 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:38.872242 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:38.872463 (+ 221us) server_negotiation.cc:299] Negotiation successful
0806 22:49:38.872587 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 22:49:38.875921 17945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:38.843690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57592 (local address 127.18.45.253:38197)
0806 22:49:38.843741 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:38.843743 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:38.845688 (+ 1945us) server_negotiation.cc:407] Connection header received
0806 22:49:38.846666 (+ 978us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:38.846667 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:38.846679 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:38.846711 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:38.859684 (+ 12973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:38.860415 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:38.863288 (+ 2873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:38.863622 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:38.863666 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:38.866556 (+ 2890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:38.866558 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:38.866559 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:38.866567 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:38.871263 (+ 4696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:38.873829 (+ 2566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:38.873832 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:38.873832 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:38.873904 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:38.874690 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:38.874691 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:38.874692 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:38.874857 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:38.874978 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:38.875714 (+ 736us) server_negotiation.cc:299] Negotiation successful
0806 22:49:38.875805 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 22:49:38.876698 17855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:38.843918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47258 (local address 127.18.45.254:44487)
0806 22:49:38.843961 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:38.843963 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:38.845590 (+ 1627us) server_negotiation.cc:407] Connection header received
0806 22:49:38.845790 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:38.845791 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:38.845802 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:38.845833 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:38.857757 (+ 11924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:38.858494 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:38.862326 (+ 3832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:38.862657 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:38.862704 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:38.866860 (+ 4156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:38.866862 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:38.866862 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:38.866871 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:38.868634 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:38.874291 (+ 5657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:38.874293 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:38.874293 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:38.874379 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:38.875065 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:38.875066 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:38.875067 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:38.875229 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:38.875316 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:38.876488 (+ 1172us) server_negotiation.cc:299] Negotiation successful
0806 22:49:38.876596 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 22:49:38.887972 18110 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:49:38.888831 18110 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:49:38.900705 18110 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 64ab4b7cbb194edf8487645652a9954e: Generated new TSK 0
I20260806 22:49:38.900982 18110 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 22:49:38.928763 17855 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:49:38.926927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47268 (local address 127.18.45.254:44487)
0806 22:49:38.926991 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:38.926994 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:38.927847 (+ 853us) server_negotiation.cc:407] Connection header received
0806 22:49:38.927986 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:38.927988 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:38.928004 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:38.928042 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:38.928657 (+ 615us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47268: BlockingRecv error: recv got EOF from 127.0.0.1:47268 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
W20260806 22:49:38.929768 17945 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:49:38.927456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57608 (local address 127.18.45.253:38197)
0806 22:49:38.927508 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:38.927510 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:38.929077 (+ 1567us) server_negotiation.cc:407] Connection header received
0806 22:49:38.929196 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:38.929198 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:38.929213 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:38.929245 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:38.929679 (+ 434us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57608: BlockingRecv error: recv got EOF from 127.0.0.1:57608 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260806 22:49:38.930961 18048 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:49:38.930176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34294 (local address 127.18.45.252:46495)
0806 22:49:38.930225 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:38.930227 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:38.930377 (+ 150us) server_negotiation.cc:407] Connection header received
0806 22:49:38.930479 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:38.930480 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:38.930493 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:38.930525 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:38.930885 (+ 360us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34294: BlockingRecv error: recv got EOF from 127.0.0.1:34294 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260806 22:49:38.940142 17945 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:49:38.935448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57620 (local address 127.18.45.253:38197)
0806 22:49:38.935500 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:38.935502 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:38.939259 (+ 3757us) server_negotiation.cc:407] Connection header received
0806 22:49:38.939417 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:38.939419 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:38.939452 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:38.939489 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:38.940028 (+ 539us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57620: BlockingRecv error: recv got EOF from 127.0.0.1:57620 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260806 22:49:38.944101 17945 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:49:38.942844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57628 (local address 127.18.45.253:38197)
0806 22:49:38.942892 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:38.942894 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:38.943393 (+ 499us) server_negotiation.cc:407] Connection header received
0806 22:49:38.943513 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:38.943515 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:38.943529 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:38.943564 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:38.944006 (+ 442us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57628: BlockingRecv error: recv got EOF from 127.0.0.1:57628 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260806 22:49:38.952010 17945 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:49:38.947545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57634 (local address 127.18.45.253:38197)
0806 22:49:38.947591 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:38.947593 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:38.951280 (+ 3687us) server_negotiation.cc:407] Connection header received
0806 22:49:38.951430 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:38.951431 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:38.951446 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:38.951478 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:38.951934 (+ 456us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57634: BlockingRecv error: recv got EOF from 127.0.0.1:57634 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260806 22:49:38.958745 17945 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:49:38.957605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57648 (local address 127.18.45.253:38197)
0806 22:49:38.957656 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:38.957659 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:38.957996 (+ 337us) server_negotiation.cc:407] Connection header received
0806 22:49:38.958114 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:38.958115 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:38.958131 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:38.958164 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:38.958659 (+ 495us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57648: BlockingRecv error: recv got EOF from 127.0.0.1:57648 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260806 22:49:38.969732 17945 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:49:38.968483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57660 (local address 127.18.45.253:38197)
0806 22:49:38.968544 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:38.968546 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:38.968952 (+ 406us) server_negotiation.cc:407] Connection header received
0806 22:49:38.969079 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:38.969080 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:38.969096 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:38.969130 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:38.969647 (+ 517us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57660: BlockingRecv error: recv got EOF from 127.0.0.1:57660 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260806 22:49:38.992187 17945 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:49:38.987483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57676 (local address 127.18.45.253:38197)
0806 22:49:38.987543 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:38.987547 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:38.991339 (+ 3792us) server_negotiation.cc:407] Connection header received
0806 22:49:38.991492 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:38.991494 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:38.991512 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:38.991547 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:38.992103 (+ 556us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57676: BlockingRecv error: recv got EOF from 127.0.0.1:57676 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260806 22:49:39.026882 17945 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:49:39.025857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57680 (local address 127.18.45.253:38197)
0806 22:49:39.025932 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:39.025934 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:39.026112 (+ 178us) server_negotiation.cc:407] Connection header received
0806 22:49:39.026208 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:39.026209 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:39.026229 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:39.026290 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:39.026783 (+ 493us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57680: BlockingRecv error: recv got EOF from 127.0.0.1:57680 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
W20260806 22:49:39.093379 17945 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:49:39.092395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57692 (local address 127.18.45.253:38197)
0806 22:49:39.092536 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:39.092538 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:39.092660 (+ 122us) server_negotiation.cc:407] Connection header received
0806 22:49:39.092733 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:39.092735 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:39.092747 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:39.092805 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:39.093297 (+ 492us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57692: BlockingRecv error: recv got EOF from 127.0.0.1:57692 (error 108)
Metrics: {"server-negotiator.queue_time_us":104}
W20260806 22:49:39.223928 17945 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:49:39.222924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57698 (local address 127.18.45.253:38197)
0806 22:49:39.223092 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:39.223095 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:39.223182 (+ 87us) server_negotiation.cc:407] Connection header received
0806 22:49:39.223305 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:39.223306 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:39.223322 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:39.223393 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:39.223836 (+ 443us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57698: BlockingRecv error: recv got EOF from 127.0.0.1:57698 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260806 22:49:39.482354 17945 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:49:39.481474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57706 (local address 127.18.45.253:38197)
0806 22:49:39.481561 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:39.481563 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:39.481591 (+ 28us) server_negotiation.cc:407] Connection header received
0806 22:49:39.481681 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:39.481683 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:39.481696 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:39.481753 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:39.482247 (+ 494us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57706: BlockingRecv error: recv got EOF from 127.0.0.1:57706 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 22:49:39.786640 18062 catalog_manager.cc:1259] Loaded cluster ID: 8cfc8e347aff4c7398370d9da16955e6
I20260806 22:49:39.786755 18062 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0e2a9a44b84c4912b44c3928f12b9b7c: loading cluster ID for follower catalog manager: success
I20260806 22:49:39.787786 18062 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0e2a9a44b84c4912b44c3928f12b9b7c: acquiring CA information for follower catalog manager: success
I20260806 22:49:39.788130 18062 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0e2a9a44b84c4912b44c3928f12b9b7c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 22:49:39.795589 18080 catalog_manager.cc:1259] Loaded cluster ID: 8cfc8e347aff4c7398370d9da16955e6
I20260806 22:49:39.795670 18080 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e23ccf2924134728a6e4518df5b3b7c8: loading cluster ID for follower catalog manager: success
I20260806 22:49:39.796519 18080 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e23ccf2924134728a6e4518df5b3b7c8: acquiring CA information for follower catalog manager: success
I20260806 22:49:39.796805 18080 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e23ccf2924134728a6e4518df5b3b7c8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 22:49:39.998834 18210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:39.995959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57722 (local address 127.18.45.253:38197)
0806 22:49:39.996120 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:39.996122 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:39.996134 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:49:39.996210 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:39.996211 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:39.996295 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:39.996373 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:49:39.996595 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:39.997176 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:39.997990 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:39.998214 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:39.998282 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:39.998468 (+ 186us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:49:39.998619 (+ 151us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:49:39.998693 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:39.998717 (+ 24us) server_negotiation.cc:299] Negotiation successful
0806 22:49:39.998744 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":84,"threads_started":1}
I20260806 22:49:40.004055 18216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:40.001089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34304 (local address 127.18.45.252:46495)
0806 22:49:40.001214 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:40.001215 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:40.001223 (+ 8us) server_negotiation.cc:407] Connection header received
0806 22:49:40.001270 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:40.001271 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:40.001319 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:40.001399 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:49:40.001544 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:40.002087 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:40.002930 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:40.003228 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:40.003289 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:40.003529 (+ 240us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:49:40.003679 (+ 150us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:49:40.003740 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:40.003835 (+ 95us) server_negotiation.cc:299] Negotiation successful
0806 22:49:40.003859 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":67,"threads_started":1}
I20260806 22:49:40.009475 18210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:40.006326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57730 (local address 127.18.45.253:38197)
0806 22:49:40.006415 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:40.006417 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:40.006573 (+ 156us) server_negotiation.cc:407] Connection header received
0806 22:49:40.006668 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:40.006670 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:40.006685 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:40.006740 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:49:40.006943 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:40.007575 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:40.008560 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:40.008884 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:40.008947 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:40.009125 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:49:40.009167 (+ 42us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:49:40.009223 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:40.009339 (+ 116us) server_negotiation.cc:299] Negotiation successful
0806 22:49:40.009378 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 22:49:40.010167 18223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:40.006329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47276 (local address 127.18.45.254:44487)
0806 22:49:40.006640 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:40.006642 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:40.006687 (+ 45us) server_negotiation.cc:407] Connection header received
0806 22:49:40.006794 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:40.006795 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:40.006850 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:40.007041 (+ 191us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:49:40.007134 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:40.007930 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:40.008877 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:40.009234 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:40.009309 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:40.009612 (+ 303us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:49:40.009770 (+ 158us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:49:40.009862 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:40.010013 (+ 151us) server_negotiation.cc:299] Negotiation successful
0806 22:49:40.010055 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":73,"threads_started":1}
I20260806 22:49:40.010288 18216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:40.006383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34308 (local address 127.18.45.252:46495)
0806 22:49:40.006432 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:40.006434 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:40.006805 (+ 371us) server_negotiation.cc:407] Connection header received
0806 22:49:40.006834 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:40.006835 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:40.006848 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:40.006904 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:49:40.007146 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:40.007927 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:40.008940 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:40.009210 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:40.009356 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:40.009908 (+ 552us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:49:40.009947 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:49:40.010000 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:40.010168 (+ 168us) server_negotiation.cc:299] Negotiation successful
0806 22:49:40.010202 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 22:49:40.012665 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 17756
I20260806 22:49:40.029750 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 17859
I20260806 22:49:40.047137 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 17948
2026-08-06T22:49:40Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3236 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 22:49:40 dist-test-slave-h3ft krb5kdc[18229](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 22:49:40 dist-test-slave-h3ft krb5kdc[18229](info): set up 2 sockets
Aug 06 22:49:40 dist-test-slave-h3ft krb5kdc[18229](info): commencing operation
krb5kdc: starting...
W20260806 22:49:41.303447 18615 mini_kdc.cc:121] Time spent starting KDC: real 1.223s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/krb5kdc/test-user.keytab.
Aug 06 22:49:41 dist-test-slave-h3ft krb5kdc[18229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056581, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T22:49:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T22:49:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.45.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab.
Entry for principal kudu/127.18.45.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab.
I20260806 22:49:41.446908 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:42385
--webserver_interface=127.18.45.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:41677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.45.254:42385,127.18.45.253:36281,127.18.45.252:43345
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 22:49:41.543536 18243 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 22:49:41.543823 18243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:41.543855 18243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:41.543926 18243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:41.545173 18243 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:49:41.545245 18243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:41.545274 18243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:41.545305 18243 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:49:41.545331 18243 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:49:41.546980 18243 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:41677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.45.254:42385,127.18.45.253:36281,127.18.45.252:43345
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.254.keytab
--principal=kudu/127.18.45.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.254:42385
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.45.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18243
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:49:41.547369 18243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:41.547660 18243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:41.553404 18253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:41.553411 18255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:41.553404 18258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:41.553629 18243 server_base.cc:1034] running on GCE node
I20260806 22:49:41.554109 18243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:41.554474 18243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:41.555652 18243 hybrid_clock.cc:648] HybridClock initialized: now 1786056581555628 us; error 36 us; skew 500 ppm
Aug 06 22:49:41 dist-test-slave-h3ft krb5kdc[18229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056581, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:41.558682 18243 init.cc:377] Logged in from keytab as kudu/127.18.45.254@KRBTEST.COM (short username kudu)
I20260806 22:49:41.559278 18243 webserver.cc:460] Webserver started at http://127.18.45.254:44905/ using document root <none> and password file <none>
I20260806 22:49:41.559567 18243 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:41.559643 18243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:41.559801 18243 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:49:41.561056 18243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/minicluster-data/master-0/data/instance:
uuid: "8a752b9c778e4c5e86ccb4389485a964"
format_stamp: "Formatted at 2026-08-06 22:49:41 on dist-test-slave-h3ft"
server_key: "47fbf8efc621ebf067b68393b004be10"
server_key_iv: "4ad3445e82c286d7e374aeb5b96f196f"
server_key_version: "encryptionkey@0"
I20260806 22:49:41.561553 18243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/minicluster-data/master-0/wal/instance:
uuid: "8a752b9c778e4c5e86ccb4389485a964"
format_stamp: "Formatted at 2026-08-06 22:49:41 on dist-test-slave-h3ft"
server_key: "47fbf8efc621ebf067b68393b004be10"
server_key_iv: "4ad3445e82c286d7e374aeb5b96f196f"
server_key_version: "encryptionkey@0"
I20260806 22:49:41.563388 18243 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.003s
I20260806 22:49:41.564260 18270 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:41.564438 18243 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:49:41.564558 18243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/minicluster-data/master-0/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/minicluster-data/master-0/wal
uuid: "8a752b9c778e4c5e86ccb4389485a964"
format_stamp: "Formatted at 2026-08-06 22:49:41 on dist-test-slave-h3ft"
server_key: "47fbf8efc621ebf067b68393b004be10"
server_key_iv: "4ad3445e82c286d7e374aeb5b96f196f"
server_key_version: "encryptionkey@0"
I20260806 22:49:41.564642 18243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:41.588879 18243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:41.591405 18243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:41.591549 18243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:41.596539 18243 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.254:42385
I20260806 22:49:41.596580 18344 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.254:42385 every 8 connection(s)
I20260806 22:49:41.597041 18243 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/minicluster-data/master-0/data/info.pb
I20260806 22:49:41.598301 18345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:41.600485 18345 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.45.254" port: 42385 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:49:41.604532 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 18243
I20260806 22:49:41.604692 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/minicluster-data/master-0/wal/instance
I20260806 22:49:41.604918 18615 external_mini_cluster.cc:1428] Setting key 6dd1d2c5ec0bc1da4d9ca9b99a2e943a
Aug 06 22:49:41 dist-test-slave-h3ft krb5kdc[18229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056581, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.254@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:49:41.612028 18351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:41.601564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40524 (local address 127.18.45.254:42385)
0806 22:49:41.601704 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:41.601709 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:41.601748 (+ 39us) server_negotiation.cc:407] Connection header received
0806 22:49:41.602439 (+ 691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:41.602441 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:41.602460 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:41.602561 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:41.603421 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.604376 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.605443 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.605789 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.605934 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:41.608484 (+ 2550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:41.608494 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:41.608494 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:41.608507 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:41.610190 (+ 1683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:41.610615 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:41.610623 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:41.610624 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:41.610706 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:41.610936 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:41.610937 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:41.610938 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:41.611250 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:41.611345 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:41.611551 (+ 206us) server_negotiation.cc:299] Negotiation successful
0806 22:49:41.611702 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":73,"threads_started":1}
I20260806 22:49:41.612039 18348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:41.601358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:42385 (local address 127.0.0.1:40524)
0806 22:49:41.601596 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:41.601648 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:41.602283 (+ 635us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:41.602582 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:41.602585 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:41.602981 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:41.603294 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:41.603301 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.604493 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.604494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:41.605356 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:41.605359 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.605906 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.605907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:41.605977 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:41.606703 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:49:41.606713 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:41.608389 (+ 1676us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:41.610289 (+ 1900us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:41.610295 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:41.610298 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:41.610517 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:41.610776 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:41.610777 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:41.610777 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:41.610871 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:41.611320 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:41.611326 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:41.611399 (+ 73us) client_negotiation.cc:769] Sending connection context
0806 22:49:41.611510 (+ 111us) client_negotiation.cc:240] Negotiation successful
0806 22:49:41.611776 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"mutex_wait_us":89,"thread_start_us":97,"threads_started":1}
I20260806 22:49:41.613083 18345 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.45.253" port: 36281 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:49:41.613585 18348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:41.613415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.253:36281 (local address 127.0.0.1:45152)
0806 22:49:41.613475 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:41.613545 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.253:36281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260806 22:49:41.613740 18275 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.45.253:36281: connect: Connection refused (error 111)
I20260806 22:49:41.614193 18348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:41.614089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.253:36281 (local address 127.0.0.1:45154)
0806 22:49:41.614127 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:41.614161 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.253:36281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
W20260806 22:49:41.614363 18345 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.45.253:36281: Network error: Client connection negotiation failed: client connection to 127.18.45.253:36281: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.18.45.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.253.keytab.
Entry for principal kudu/127.18.45.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.253.keytab.
I20260806 22:49:41.637794 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.253:36281
--webserver_interface=127.18.45.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.253.keytab
--principal=kudu/127.18.45.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:41677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.45.254:42385,127.18.45.253:36281,127.18.45.252:43345
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:49:41.663681 18345 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.45.253" port: 36281 } attempt: 1
I20260806 22:49:41.664441 18348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:41.664232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.253:36281 (local address 127.0.0.1:45156)
0806 22:49:41.664315 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:41.664371 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.253:36281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260806 22:49:41.664867 18348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:41.664774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.253:36281 (local address 127.0.0.1:45162)
0806 22:49:41.664818 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:41.664846 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.253:36281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260806 22:49:41.664973 18345 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.45.253:36281: Network error: Client connection negotiation failed: client connection to 127.18.45.253:36281: connect: Connection refused (error 111)
I20260806 22:49:41.733141 18345 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.45.253" port: 36281 } attempt: 2
I20260806 22:49:41.733740 18348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:41.733577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.253:36281 (local address 127.0.0.1:45176)
0806 22:49:41.733649 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:41.733698 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.253:36281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260806 22:49:41.734153 18348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:41.734045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.253:36281 (local address 127.0.0.1:45190)
0806 22:49:41.734092 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:41.734120 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.253:36281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260806 22:49:41.734246 18345 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.45.253:36281: Network error: Client connection negotiation failed: client connection to 127.18.45.253:36281: connect: Connection refused (error 111)
W20260806 22:49:41.738689 18356 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 22:49:41.738953 18356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:41.738973 18356 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:41.739092 18356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:41.740396 18356 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:49:41.740444 18356 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:41.740459 18356 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:41.740473 18356 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:49:41.740522 18356 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:49:41.742245 18356 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:41677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.18.45.254:42385,127.18.45.253:36281,127.18.45.252:43345
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.253.keytab
--principal=kudu/127.18.45.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.253:36281
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.45.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18356
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:49:41.742564 18356 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:41.742843 18356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:41.748689 18365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:41.748661 18364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:41.749073 18356 server_base.cc:1034] running on GCE node
W20260806 22:49:41.748667 18367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:41.749421 18356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:41.749817 18356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:41.750994 18356 hybrid_clock.cc:648] HybridClock initialized: now 1786056581750973 us; error 40 us; skew 500 ppm
Aug 06 22:49:41 dist-test-slave-h3ft krb5kdc[18229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056581, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:41.754235 18356 init.cc:377] Logged in from keytab as kudu/127.18.45.253@KRBTEST.COM (short username kudu)
I20260806 22:49:41.754860 18356 webserver.cc:460] Webserver started at http://127.18.45.253:34071/ using document root <none> and password file <none>
I20260806 22:49:41.755192 18356 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:41.755270 18356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:41.755445 18356 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:49:41.756883 18356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/minicluster-data/master-1/data/instance:
uuid: "f08a2826c5774e009b88d7ac67a4ec8e"
format_stamp: "Formatted at 2026-08-06 22:49:41 on dist-test-slave-h3ft"
server_key: "9dfa464c9d9e174bf03636699f7b4381"
server_key_iv: "368699d35684351baf4bb2f81fb72496"
server_key_version: "encryptionkey@0"
I20260806 22:49:41.757342 18356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/minicluster-data/master-1/wal/instance:
uuid: "f08a2826c5774e009b88d7ac67a4ec8e"
format_stamp: "Formatted at 2026-08-06 22:49:41 on dist-test-slave-h3ft"
server_key: "9dfa464c9d9e174bf03636699f7b4381"
server_key_iv: "368699d35684351baf4bb2f81fb72496"
server_key_version: "encryptionkey@0"
I20260806 22:49:41.758920 18356 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260806 22:49:41.759913 18377 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:41.760165 18356 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 22:49:41.760291 18356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/minicluster-data/master-1/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/minicluster-data/master-1/wal
uuid: "f08a2826c5774e009b88d7ac67a4ec8e"
format_stamp: "Formatted at 2026-08-06 22:49:41 on dist-test-slave-h3ft"
server_key: "9dfa464c9d9e174bf03636699f7b4381"
server_key_iv: "368699d35684351baf4bb2f81fb72496"
server_key_version: "encryptionkey@0"
I20260806 22:49:41.760375 18356 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:41.776142 18356 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:41.778925 18356 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:41.779145 18356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:41.784312 18356 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.253:36281
I20260806 22:49:41.784358 18452 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.253:36281 every 8 connection(s)
I20260806 22:49:41.784824 18356 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/minicluster-data/master-1/data/info.pb
I20260806 22:49:41.785769 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 18356
I20260806 22:49:41.785919 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/minicluster-data/master-1/wal/instance
I20260806 22:49:41.786213 18615 external_mini_cluster.cc:1428] Setting key b7d06c66b7b43d61da1c1c43b55169ab
I20260806 22:49:41.786444 18453 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:41.789608 18453 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.45.254" port: 42385 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.18.45.252@KRBTEST.COM; defaulting to no policy
Aug 06 22:49:41 dist-test-slave-h3ft krb5kdc[18229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056581, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.253@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.45.252@KRBTEST.COM" created.
I20260806 22:49:41.801318 18351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:41.790619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40528 (local address 127.18.45.254:42385)
0806 22:49:41.790701 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:41.790704 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:41.790987 (+ 283us) server_negotiation.cc:407] Connection header received
0806 22:49:41.791712 (+ 725us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:41.791714 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:41.791736 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:41.791839 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:41.792799 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.793423 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.794625 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.794891 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.794980 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:41.797694 (+ 2714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:41.797697 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:41.797697 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:41.797714 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:41.799443 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:41.800022 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:41.800024 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:41.800024 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:41.800091 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:41.800460 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:41.800461 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:41.800462 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:41.800627 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:41.800708 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:41.801125 (+ 417us) server_negotiation.cc:299] Negotiation successful
0806 22:49:41.801224 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 22:49:41.801681 18457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:41.790578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:42385 (local address 127.0.0.1:40528)
0806 22:49:41.790823 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:41.790869 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:41.791574 (+ 705us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:41.791939 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:41.791943 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:41.792347 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:41.792688 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:41.792695 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.793546 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.793547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:41.794509 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:41.794512 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.795050 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.795053 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:41.795199 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:41.795954 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:49:41.795965 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:41.797575 (+ 1610us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:41.799612 (+ 2037us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:41.799622 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:41.799625 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:41.799947 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:41.800158 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:41.800159 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:41.800159 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:41.800332 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:41.800724 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:41.800732 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:41.800942 (+ 210us) client_negotiation.cc:769] Sending connection context
0806 22:49:41.801085 (+ 143us) client_negotiation.cc:240] Negotiation successful
0806 22:49:41.801311 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":114,"threads_started":1}
I20260806 22:49:41.802464 18453 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.45.253" port: 36281 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:49:41 dist-test-slave-h3ft krb5kdc[18229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056581, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.253@KRBTEST.COM for kudu/127.18.45.253@KRBTEST.COM
I20260806 22:49:41.812218 18457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:41.802836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.253:36281 (local address 127.0.0.1:45196)
0806 22:49:41.802933 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:41.802948 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:41.803022 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:41.803377 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:41.803378 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:41.803528 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:41.803609 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:41.803610 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.804921 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.804922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:41.805507 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:41.805508 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.806272 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.806273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:41.806340 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:41.806887 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:49:41.806893 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:41.808652 (+ 1759us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:41.811062 (+ 2410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:41.811062 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:41.811063 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:41.811261 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:41.811499 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:41.811500 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:41.811500 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:41.811561 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:41.811931 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:41.811931 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:41.811975 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 22:49:41.812027 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 22:49:41.812127 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72}
I20260806 22:49:41.812299 18460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:41.802931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45196 (local address 127.18.45.253:36281)
0806 22:49:41.803184 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:41.803191 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:41.803220 (+ 29us) server_negotiation.cc:407] Connection header received
0806 22:49:41.803246 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:41.803248 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:41.803267 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:41.803361 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:41.803688 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.804796 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.805590 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.806184 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.806267 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:41.808745 (+ 2478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:41.808754 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:41.808755 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:41.808763 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:41.810925 (+ 2162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:41.811345 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:41.811349 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:41.811350 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:41.811422 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:41.811634 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:41.811635 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:41.811636 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:41.811860 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:41.811965 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:41.812061 (+ 96us) server_negotiation.cc:299] Negotiation successful
0806 22:49:41.812193 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":214,"thread_start_us":166,"threads_started":1}
I20260806 22:49:41.813031 18453 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.45.252" port: 43345 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:49:41.813472 18457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:41.813344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.252:43345 (local address 127.0.0.1:49916)
0806 22:49:41.813393 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:41.813449 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.252:43345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260806 22:49:41.813673 18383 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.45.252:43345: connect: Connection refused (error 111)
I20260806 22:49:41.814234 18457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:41.814123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.252:43345 (local address 127.0.0.1:49922)
0806 22:49:41.814175 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:41.814210 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.252:43345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260806 22:49:41.814373 18453 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.45.252:43345: Network error: Client connection negotiation failed: client connection to 127.18.45.252:43345: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.45.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.252.keytab.
Entry for principal kudu/127.18.45.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.252.keytab.
I20260806 22:49:41.821825 18615 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXSWSdD/build/release/bin/kudu
/tmp/dist-test-taskXSWSdD/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.252:43345
--webserver_interface=127.18.45.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.252.keytab
--principal=kudu/127.18.45.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.45.212:41677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.45.254:42385,127.18.45.253:36281,127.18.45.252:43345
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 22:49:41.825403 18345 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.45.253" port: 36281 } attempt: 3
Aug 06 22:49:41 dist-test-slave-h3ft krb5kdc[18229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056581, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.254@KRBTEST.COM for kudu/127.18.45.253@KRBTEST.COM
I20260806 22:49:41.834219 18348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:41.825807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.253:36281 (local address 127.0.0.1:45208)
0806 22:49:41.825884 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:41.825919 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:41.825987 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:41.826169 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:41.826170 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:41.826380 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:41.826475 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:41.826478 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.827348 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.827350 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:41.827991 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:41.827993 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.828442 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.828443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:41.828542 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:41.829112 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:49:41.829118 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:41.830915 (+ 1797us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:41.832961 (+ 2046us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:41.832962 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:41.832962 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:41.833185 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:41.833430 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:41.833431 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:41.833431 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:41.833514 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:41.833805 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:41.833806 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:41.833876 (+ 70us) client_negotiation.cc:769] Sending connection context
0806 22:49:41.833964 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 22:49:41.834091 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260806 22:49:41.834219 18460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:41.825880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45208 (local address 127.18.45.253:36281)
0806 22:49:41.825954 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:41.825957 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:41.825970 (+ 13us) server_negotiation.cc:407] Connection header received
0806 22:49:41.826075 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:41.826076 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:41.826088 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:41.826162 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:41.826571 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.827255 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.828087 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.828352 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.828444 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:41.831046 (+ 2602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:41.831048 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:41.831048 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:41.831058 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:41.832821 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:41.833272 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:41.833273 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:41.833274 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:41.833348 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:41.833597 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:41.833598 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:41.833598 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:41.833733 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:41.833815 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:41.833994 (+ 179us) server_negotiation.cc:299] Negotiation successful
0806 22:49:41.834092 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 22:49:41.834900 18345 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.45.252" port: 43345 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:49:41.835390 18348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:41.835253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.252:43345 (local address 127.0.0.1:49938)
0806 22:49:41.835311 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:41.835344 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.252:43345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260806 22:49:41.835781 18348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:41.835714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.252:43345 (local address 127.0.0.1:49948)
0806 22:49:41.835742 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:41.835766 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.252:43345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260806 22:49:41.835881 18345 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.45.252:43345: Network error: Client connection negotiation failed: client connection to 127.18.45.252:43345: connect: Connection refused (error 111)
I20260806 22:49:41.863641 18453 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.45.252" port: 43345 } attempt: 1
I20260806 22:49:41.864351 18457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:41.864170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.252:43345 (local address 127.0.0.1:49952)
0806 22:49:41.864240 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:41.864299 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.252:43345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260806 22:49:41.864868 18457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:41.864750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.252:43345 (local address 127.0.0.1:49958)
0806 22:49:41.864805 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:41.864839 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.252:43345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260806 22:49:41.865053 18453 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.45.252:43345: Network error: Client connection negotiation failed: client connection to 127.18.45.252:43345: connect: Connection refused (error 111)
I20260806 22:49:41.867079 18345 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.45.252" port: 43345 } attempt: 1
I20260806 22:49:41.867547 18348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:41.867408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.252:43345 (local address 127.0.0.1:49968)
0806 22:49:41.867478 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:41.867516 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.252:43345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260806 22:49:41.868014 18348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:41.867845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.252:43345 (local address 127.0.0.1:49980)
0806 22:49:41.867952 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:41.867982 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.252:43345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
W20260806 22:49:41.868124 18345 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.45.252:43345: Network error: Client connection negotiation failed: client connection to 127.18.45.252:43345: connect: Connection refused (error 111)
W20260806 22:49:41.920909 18462 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 22:49:41.921226 18462 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 22:49:41.921257 18462 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 22:49:41.921293 18462 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 22:49:41.922516 18462 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 22:49:41.922585 18462 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 22:49:41.922609 18462 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 22:49:41.922631 18462 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 22:49:41.922668 18462 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 22:49:41.924335 18462 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.45.212:41677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.18.45.254:42385,127.18.45.253:36281,127.18.45.252:43345
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/krb5kdc/kudu_127.18.45.252.keytab
--principal=kudu/127.18.45.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.45.252:43345
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.45.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18462
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 21:52:56 UTC on ccycloud.quasar-zvmlvv.root.comops.site
build id 4571540
I20260806 22:49:41.924646 18462 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 22:49:41.924924 18462 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 22:49:41.930735 18471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 22:49:41.930923 18469 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:41.931113 18462 server_base.cc:1034] running on GCE node
W20260806 22:49:41.930941 18475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 22:49:41.931444 18462 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 22:49:41.931792 18462 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 22:49:41.932989 18462 hybrid_clock.cc:648] HybridClock initialized: now 1786056581932970 us; error 37 us; skew 500 ppm
I20260806 22:49:41.933266 18453 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.45.252" port: 43345 } attempt: 2
I20260806 22:49:41.933802 18457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:41.933651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.252:43345 (local address 127.0.0.1:49986)
0806 22:49:41.933718 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:41.933758 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.252:43345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 22:49:41.934198 18457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:41.934094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.252:43345 (local address 127.0.0.1:49992)
0806 22:49:41.934142 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:41.934175 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.252:43345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260806 22:49:41.934288 18453 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.45.252:43345: Network error: Client connection negotiation failed: client connection to 127.18.45.252:43345: connect: Connection refused (error 111)
Aug 06 22:49:41 dist-test-slave-h3ft krb5kdc[18229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056581, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 22:49:41.936259 18462 init.cc:377] Logged in from keytab as kudu/127.18.45.252@KRBTEST.COM (short username kudu)
I20260806 22:49:41.936714 18462 webserver.cc:460] Webserver started at http://127.18.45.252:33643/ using document root <none> and password file <none>
I20260806 22:49:41.936928 18462 fs_manager.cc:359] Metadata directory not provided
I20260806 22:49:41.936969 18462 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 22:49:41.937089 18462 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 22:49:41.938297 18462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/minicluster-data/master-2/data/instance:
uuid: "10d437a3ad0141ef936f6ff6ce557bf9"
format_stamp: "Formatted at 2026-08-06 22:49:41 on dist-test-slave-h3ft"
server_key: "02591baa48a262cab6d3b175de93c2e2"
server_key_iv: "f7991db9a8cd4b89cc44b1b349e1b435"
server_key_version: "encryptionkey@0"
I20260806 22:49:41.938706 18462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/minicluster-data/master-2/wal/instance:
uuid: "10d437a3ad0141ef936f6ff6ce557bf9"
format_stamp: "Formatted at 2026-08-06 22:49:41 on dist-test-slave-h3ft"
server_key: "02591baa48a262cab6d3b175de93c2e2"
server_key_iv: "f7991db9a8cd4b89cc44b1b349e1b435"
server_key_version: "encryptionkey@0"
I20260806 22:49:41.940232 18462 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 22:49:41.940979 18482 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:41.941162 18462 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 22:49:41.941273 18462 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/minicluster-data/master-2/data,/tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/minicluster-data/master-2/wal
uuid: "10d437a3ad0141ef936f6ff6ce557bf9"
format_stamp: "Formatted at 2026-08-06 22:49:41 on dist-test-slave-h3ft"
server_key: "02591baa48a262cab6d3b175de93c2e2"
server_key_iv: "f7991db9a8cd4b89cc44b1b349e1b435"
server_key_version: "encryptionkey@0"
I20260806 22:49:41.941351 18462 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 22:49:41.943275 18345 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.45.252" port: 43345 } attempt: 2
I20260806 22:49:41.943711 18348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:41.943578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.252:43345 (local address 127.0.0.1:50002)
0806 22:49:41.943639 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:41.943676 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.252:43345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 22:49:41.944095 18348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:41.943976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.252:43345 (local address 127.0.0.1:50016)
0806 22:49:41.944044 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:41.944074 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.45.252:43345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260806 22:49:41.944185 18345 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.45.252:43345: Network error: Client connection negotiation failed: client connection to 127.18.45.252:43345: connect: Connection refused (error 111)
I20260806 22:49:41.956869 18462 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 22:49:41.959571 18462 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 22:49:41.959779 18462 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 22:49:41.965466 18462 rpc_server.cc:307] RPC server started. Bound to: 127.18.45.252:43345
I20260806 22:49:41.965521 18553 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.45.252:43345 every 8 connection(s)
I20260806 22:49:41.965966 18462 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/minicluster-data/master-2/data/info.pb
I20260806 22:49:41.967407 18554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 22:49:41.969205 18615 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXSWSdD/build/release/bin/kudu as pid 18462
I20260806 22:49:41.969314 18615 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXSWSdD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786056275803658-18615-0/minicluster-data/master-2/wal/instance
I20260806 22:49:41.969687 18615 external_mini_cluster.cc:1428] Setting key 28733180628848e09cf99b5ff4b9e8c8
I20260806 22:49:41.969655 18554 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.45.254" port: 42385 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 22:49:41 dist-test-slave-h3ft krb5kdc[18229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056581, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.252@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
Aug 06 22:49:41 dist-test-slave-h3ft krb5kdc[18229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056581, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.45.252@KRBTEST.COM
I20260806 22:49:41.980708 18351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:41.970511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40542 (local address 127.18.45.254:42385)
0806 22:49:41.970597 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:41.970599 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:41.970777 (+ 178us) server_negotiation.cc:407] Connection header received
0806 22:49:41.971458 (+ 681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:41.971459 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:41.971475 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:41.971568 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:41.972200 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.972727 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.973824 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.974131 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.974210 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:41.977176 (+ 2966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:41.977178 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:41.977178 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:41.977201 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:41.979156 (+ 1955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:41.979648 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:41.979650 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:41.979651 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:41.979728 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:41.979989 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:41.979990 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:41.979991 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:41.980128 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:41.980205 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:41.980488 (+ 283us) server_negotiation.cc:299] Negotiation successful
0806 22:49:41.980598 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 22:49:41.980885 18562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:41.970454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.254:42385 (local address 127.0.0.1:40542)
0806 22:49:41.970628 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:41.970678 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:41.971321 (+ 643us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:41.971592 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:41.971594 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:41.971910 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:41.972101 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:41.972105 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.972803 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.972804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:41.973751 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:41.973753 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.974258 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.974259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:41.974436 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:41.975218 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:49:41.975239 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:41.977086 (+ 1847us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:41.979239 (+ 2153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:41.979245 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:41.979248 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:41.979576 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:41.979801 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:41.979802 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:41.979802 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:41.979907 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:41.980195 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:41.980203 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:41.980345 (+ 142us) client_negotiation.cc:769] Sending connection context
0806 22:49:41.980454 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 22:49:41.980620 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":112,"threads_started":1}
I20260806 22:49:41.981539 18554 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.45.253" port: 36281 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:49:41.981673 18560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:41.970318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50024 (local address 127.18.45.252:43345)
0806 22:49:41.970640 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:41.970651 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:41.970687 (+ 36us) server_negotiation.cc:407] Connection header received
0806 22:49:41.971321 (+ 634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:41.971332 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:41.971362 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:41.971465 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:41.972233 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.973342 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.974334 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.974767 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.974831 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:41.977704 (+ 2873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:41.977712 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:41.977712 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:41.977719 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:41.979845 (+ 2126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:41.980224 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:41.980228 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:41.980228 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:41.980297 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:41.980579 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:41.980580 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:41.980581 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:41.980752 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:41.981420 (+ 668us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:41.981477 (+ 57us) server_negotiation.cc:299] Negotiation successful
0806 22:49:41.981568 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":105,"threads_started":1}
Aug 06 22:49:41 dist-test-slave-h3ft krb5kdc[18229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056581, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.252@KRBTEST.COM for kudu/127.18.45.253@KRBTEST.COM
Aug 06 22:49:41 dist-test-slave-h3ft krb5kdc[18229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056581, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.45.254@KRBTEST.COM
I20260806 22:49:41.989464 18562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:41.981898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.253:36281 (local address 127.0.0.1:45220)
0806 22:49:41.981950 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:41.981963 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:41.982007 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:41.982178 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:41.982179 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:41.982349 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:41.982406 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:41.982408 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.983269 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.983271 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:41.983863 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:41.983866 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.984366 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.984367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:41.984438 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:41.984848 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:49:41.984854 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:41.986341 (+ 1487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:41.988312 (+ 1971us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:41.988313 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:41.988313 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:41.988502 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:41.988739 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:41.988740 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:41.988740 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:41.988805 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:41.989113 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:41.989114 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:41.989163 (+ 49us) client_negotiation.cc:769] Sending connection context
0806 22:49:41.989245 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 22:49:41.989360 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 22:49:41.989532 18460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:41.981981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45220 (local address 127.18.45.253:36281)
0806 22:49:41.982040 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:41.982042 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:41.982052 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:49:41.982088 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:41.982089 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:41.982103 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:41.982180 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:41.982492 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.983189 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.983955 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.984292 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.984353 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:41.986444 (+ 2091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:41.986445 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:41.986446 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:41.986456 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:41.988220 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:41.988592 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:41.988593 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:41.988593 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:41.988662 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:41.988890 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:41.988892 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:41.988892 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:41.989032 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:41.989110 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:41.989294 (+ 184us) server_negotiation.cc:299] Negotiation successful
0806 22:49:41.989404 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 22:49:41.990093 18554 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.45.252" port: 43345 } has no permanent_uuid. Determining permanent_uuid...
I20260806 22:49:41.990841 18351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:41.982675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40550 (local address 127.18.45.254:42385)
0806 22:49:41.982714 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:41.982716 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:41.982762 (+ 46us) server_negotiation.cc:407] Connection header received
0806 22:49:41.982834 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:41.982835 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:41.982844 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:41.982892 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:41.983488 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.984057 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.984919 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.985119 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.985175 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:41.987826 (+ 2651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:41.987827 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:41.987827 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:41.987833 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:41.989220 (+ 1387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:41.989607 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:41.989609 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:41.989609 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:41.989685 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:41.990088 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:41.990089 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:41.990089 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:41.990235 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:41.990436 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:41.990630 (+ 194us) server_negotiation.cc:299] Negotiation successful
0806 22:49:41.990720 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
Aug 06 22:49:41 dist-test-slave-h3ft krb5kdc[18229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056581, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.252@KRBTEST.COM for kudu/127.18.45.252@KRBTEST.COM
Aug 06 22:49:41 dist-test-slave-h3ft krb5kdc[18229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056581, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.45.253@KRBTEST.COM
I20260806 22:49:41.997862 18560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:41.990437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50034 (local address 127.18.45.252:43345)
0806 22:49:41.990473 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:41.990475 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:41.990484 (+ 9us) server_negotiation.cc:407] Connection header received
0806 22:49:41.990561 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:41.990562 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:41.990571 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:41.990620 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:41.990935 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.991637 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.992573 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.992891 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.992964 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:41.995142 (+ 2178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:41.995144 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:41.995144 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:41.995150 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:41.996666 (+ 1516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:41.997020 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:41.997021 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:41.997021 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:41.997084 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:41.997282 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:41.997283 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:41.997283 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:41.997397 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:41.997467 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:41.997621 (+ 154us) server_negotiation.cc:299] Negotiation successful
0806 22:49:41.997767 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":72,"server-negotiator.queue_time_us":19}
I20260806 22:49:41.997861 18562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:41.990383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.252:43345 (local address 127.0.0.1:50034)
0806 22:49:41.990437 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:41.990448 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:41.990500 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:41.990657 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:41.990657 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:41.990818 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:41.990849 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:41.990851 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.991721 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.991722 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:41.992478 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:41.992480 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.992975 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.992976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:41.993015 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:41.993416 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:49:41.993421 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:41.994992 (+ 1571us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:41.996745 (+ 1753us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:41.996747 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:41.996747 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:41.996944 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:41.997144 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:41.997145 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:41.997146 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:41.997202 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:41.997469 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:41.997470 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:41.997517 (+ 47us) client_negotiation.cc:769] Sending connection context
0806 22:49:41.997598 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 22:49:41.997714 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260806 22:49:41.999855 18460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:41.991638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45234 (local address 127.18.45.253:36281)
0806 22:49:41.991676 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:41.991678 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:41.991690 (+ 12us) server_negotiation.cc:407] Connection header received
0806 22:49:41.991715 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:41.991716 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:41.991730 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:41.991779 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:41.992352 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.993034 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.993969 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:41.994262 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:41.994344 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:41.996855 (+ 2511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:41.996856 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:41.996857 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:41.996864 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:41.998583 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:41.998931 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:41.998932 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:41.998933 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:41.999070 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:41.999270 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:41.999271 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:41.999272 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:41.999418 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:41.999626 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:41.999669 (+ 43us) server_negotiation.cc:299] Negotiation successful
0806 22:49:41.999757 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 22:49:42.000180 18554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10d437a3ad0141ef936f6ff6ce557bf9: Bootstrap starting.
I20260806 22:49:42.001516 18554 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 10d437a3ad0141ef936f6ff6ce557bf9: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:42.001718 18615 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 22:49:42.001886 18554 log.cc:824] T 00000000000000000000000000000000 P 10d437a3ad0141ef936f6ff6ce557bf9: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:42.002811 18554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10d437a3ad0141ef936f6ff6ce557bf9: No bootstrap required, opened a new log
I20260806 22:49:42.003114 18311 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "8a752b9c778e4c5e86ccb4389485a964"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40550
I20260806 22:49:42.005297 18554 raft_consensus.cc:348] T 00000000000000000000000000000000 P 10d437a3ad0141ef936f6ff6ce557bf9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a752b9c778e4c5e86ccb4389485a964" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 42385 } } peers { permanent_uuid: "f08a2826c5774e009b88d7ac67a4ec8e" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 36281 } } peers { permanent_uuid: "10d437a3ad0141ef936f6ff6ce557bf9" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 43345 } }
I20260806 22:49:42.005479 18554 raft_consensus.cc:729] T 00000000000000000000000000000000 P 10d437a3ad0141ef936f6ff6ce557bf9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10d437a3ad0141ef936f6ff6ce557bf9, State: Initialized, Role: FOLLOWER
I20260806 22:49:42.005587 18554 consensus_queue.cc:260] T 00000000000000000000000000000000 P 10d437a3ad0141ef936f6ff6ce557bf9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a752b9c778e4c5e86ccb4389485a964" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 42385 } } peers { permanent_uuid: "f08a2826c5774e009b88d7ac67a4ec8e" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 36281 } } peers { permanent_uuid: "10d437a3ad0141ef936f6ff6ce557bf9" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 43345 } }
I20260806 22:49:42.005931 18564 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10d437a3ad0141ef936f6ff6ce557bf9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a752b9c778e4c5e86ccb4389485a964" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 42385 } } peers { permanent_uuid: "f08a2826c5774e009b88d7ac67a4ec8e" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 36281 } } peers { permanent_uuid: "10d437a3ad0141ef936f6ff6ce557bf9" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 43345 } } }
I20260806 22:49:42.006003 18554 sys_catalog.cc:564] T 00000000000000000000000000000000 P 10d437a3ad0141ef936f6ff6ce557bf9 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:49:42.006065 18564 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10d437a3ad0141ef936f6ff6ce557bf9 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 22:49:42.008176 18578 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 10d437a3ad0141ef936f6ff6ce557bf9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:49:42.008240 18578 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:49:42.008507 18462 master_runner.cc:186] Error getting master registration for 127.18.45.254:42385: OK, instance_id { permanent_uuid: "8a752b9c778e4c5e86ccb4389485a964" instance_seqno: 1786056581595068 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 22:49:42.011214 18462 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 22:49:42.025447 18453 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.45.252" port: 43345 } attempt: 3
Aug 06 22:49:42 dist-test-slave-h3ft krb5kdc[18229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056581, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.253@KRBTEST.COM for kudu/127.18.45.252@KRBTEST.COM
I20260806 22:49:42.034070 18457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:42.025941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.252:43345 (local address 127.0.0.1:50044)
0806 22:49:42.026023 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:42.026039 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:42.026106 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:42.026265 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:42.026267 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:42.026482 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:42.026549 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:42.026551 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:42.027314 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:42.027316 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:42.027858 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:42.027859 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:42.028234 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:42.028235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:42.028298 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:42.028881 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:49:42.028887 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:42.030737 (+ 1850us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:42.032792 (+ 2055us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:42.032793 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:42.032794 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:42.033048 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:42.033260 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:42.033261 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:42.033261 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:42.033333 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:42.033702 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:42.033703 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:42.033755 (+ 52us) client_negotiation.cc:769] Sending connection context
0806 22:49:42.033830 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 22:49:42.033944 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260806 22:49:42.034068 18560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:42.025987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50044 (local address 127.18.45.252:43345)
0806 22:49:42.026044 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:42.026046 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:42.026087 (+ 41us) server_negotiation.cc:407] Connection header received
0806 22:49:42.026169 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:42.026170 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:42.026188 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:42.026258 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:42.026655 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:42.027229 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:42.027950 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:42.028150 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:42.028227 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:42.030845 (+ 2618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:42.030848 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:42.030853 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:42.030861 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:42.032700 (+ 1839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:42.033130 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:42.033131 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:42.033131 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:42.033197 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:42.033399 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:42.033400 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:42.033400 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:42.033634 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:42.033704 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:42.033849 (+ 145us) server_negotiation.cc:299] Negotiation successful
0806 22:49:42.033946 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 22:49:42.036762 18453 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f08a2826c5774e009b88d7ac67a4ec8e: Bootstrap starting.
I20260806 22:49:42.037845 18453 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f08a2826c5774e009b88d7ac67a4ec8e: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:42.038237 18453 log.cc:824] T 00000000000000000000000000000000 P f08a2826c5774e009b88d7ac67a4ec8e: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:42.039314 18453 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f08a2826c5774e009b88d7ac67a4ec8e: No bootstrap required, opened a new log
I20260806 22:49:42.041641 18453 raft_consensus.cc:348] T 00000000000000000000000000000000 P f08a2826c5774e009b88d7ac67a4ec8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a752b9c778e4c5e86ccb4389485a964" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 42385 } } peers { permanent_uuid: "f08a2826c5774e009b88d7ac67a4ec8e" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 36281 } } peers { permanent_uuid: "10d437a3ad0141ef936f6ff6ce557bf9" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 43345 } }
I20260806 22:49:42.041817 18453 raft_consensus.cc:729] T 00000000000000000000000000000000 P f08a2826c5774e009b88d7ac67a4ec8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f08a2826c5774e009b88d7ac67a4ec8e, State: Initialized, Role: FOLLOWER
I20260806 22:49:42.041914 18453 consensus_queue.cc:260] T 00000000000000000000000000000000 P f08a2826c5774e009b88d7ac67a4ec8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a752b9c778e4c5e86ccb4389485a964" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 42385 } } peers { permanent_uuid: "f08a2826c5774e009b88d7ac67a4ec8e" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 36281 } } peers { permanent_uuid: "10d437a3ad0141ef936f6ff6ce557bf9" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 43345 } }
I20260806 22:49:42.042276 18581 sys_catalog.cc:455] T 00000000000000000000000000000000 P f08a2826c5774e009b88d7ac67a4ec8e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a752b9c778e4c5e86ccb4389485a964" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 42385 } } peers { permanent_uuid: "f08a2826c5774e009b88d7ac67a4ec8e" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 36281 } } peers { permanent_uuid: "10d437a3ad0141ef936f6ff6ce557bf9" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 43345 } } }
I20260806 22:49:42.042346 18453 sys_catalog.cc:564] T 00000000000000000000000000000000 P f08a2826c5774e009b88d7ac67a4ec8e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:49:42.042382 18581 sys_catalog.cc:458] T 00000000000000000000000000000000 P f08a2826c5774e009b88d7ac67a4ec8e [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:49:42.043313 18345 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.45.252" port: 43345 } attempt: 3
W20260806 22:49:42.044955 18600 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f08a2826c5774e009b88d7ac67a4ec8e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:49:42.045014 18600 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:49:42.045436 18356 master_runner.cc:186] Error getting master registration for 127.18.45.254:42385: OK, instance_id { permanent_uuid: "8a752b9c778e4c5e86ccb4389485a964" instance_seqno: 1786056581595068 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 22:49:42.048657 18356 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 06 22:49:42 dist-test-slave-h3ft krb5kdc[18229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786056581, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.45.254@KRBTEST.COM for kudu/127.18.45.252@KRBTEST.COM
I20260806 22:49:42.052174 18348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:42.043698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.45.252:43345 (local address 127.0.0.1:50050)
0806 22:49:42.043776 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0806 22:49:42.043788 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 22:49:42.043833 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 22:49:42.043998 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 22:49:42.044011 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 22:49:42.044249 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0806 22:49:42.044318 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:42.044322 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:42.045281 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:42.045282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:42.046128 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 22:49:42.046130 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 22:49:42.046673 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 22:49:42.046674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 22:49:42.046740 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:42.047346 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 22:49:42.047353 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 22:49:42.049021 (+ 1668us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 22:49:42.050939 (+ 1918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:42.050940 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:42.050941 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:42.051228 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:42.051449 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 22:49:42.051449 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 22:49:42.051450 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 22:49:42.051517 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 22:49:42.051796 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 22:49:42.051797 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 22:49:42.051849 (+ 52us) client_negotiation.cc:769] Sending connection context
0806 22:49:42.051932 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 22:49:42.052063 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260806 22:49:42.052173 18560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:42.043782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50050 (local address 127.18.45.252:43345)
0806 22:49:42.043833 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:42.043835 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:42.043876 (+ 41us) server_negotiation.cc:407] Connection header received
0806 22:49:42.043903 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:42.043904 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:42.043917 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:42.043976 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:42.044427 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:42.045170 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:42.046248 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:42.046550 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:42.046611 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:42.049102 (+ 2491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:42.049104 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:42.049104 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:42.049113 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:42.050834 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:42.051301 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:42.051302 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:42.051303 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:42.051374 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:42.051572 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:42.051573 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:42.051574 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:42.051739 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:42.051804 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:42.051975 (+ 171us) server_negotiation.cc:299] Negotiation successful
0806 22:49:42.052063 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 22:49:42.054330 18345 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a752b9c778e4c5e86ccb4389485a964: Bootstrap starting.
I20260806 22:49:42.055223 18345 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8a752b9c778e4c5e86ccb4389485a964: Neither blocks nor log segments found. Creating new log.
I20260806 22:49:42.055578 18345 log.cc:824] T 00000000000000000000000000000000 P 8a752b9c778e4c5e86ccb4389485a964: Log is configured to *not* fsync() on all Append() calls
I20260806 22:49:42.056520 18345 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a752b9c778e4c5e86ccb4389485a964: No bootstrap required, opened a new log
I20260806 22:49:42.058100 18345 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a752b9c778e4c5e86ccb4389485a964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a752b9c778e4c5e86ccb4389485a964" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 42385 } } peers { permanent_uuid: "f08a2826c5774e009b88d7ac67a4ec8e" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 36281 } } peers { permanent_uuid: "10d437a3ad0141ef936f6ff6ce557bf9" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 43345 } }
I20260806 22:49:42.058257 18345 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a752b9c778e4c5e86ccb4389485a964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a752b9c778e4c5e86ccb4389485a964, State: Initialized, Role: FOLLOWER
I20260806 22:49:42.058355 18345 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a752b9c778e4c5e86ccb4389485a964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a752b9c778e4c5e86ccb4389485a964" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 42385 } } peers { permanent_uuid: "f08a2826c5774e009b88d7ac67a4ec8e" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 36281 } } peers { permanent_uuid: "10d437a3ad0141ef936f6ff6ce557bf9" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 43345 } }
I20260806 22:49:42.058737 18345 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a752b9c778e4c5e86ccb4389485a964 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 22:49:42.058832 18311 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a752b9c778e4c5e86ccb4389485a964 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 22:49:42.058887 18311 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a752b9c778e4c5e86ccb4389485a964 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:42.059971 18311 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a752b9c778e4c5e86ccb4389485a964 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a752b9c778e4c5e86ccb4389485a964" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 42385 } } peers { permanent_uuid: "f08a2826c5774e009b88d7ac67a4ec8e" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 36281 } } peers { permanent_uuid: "10d437a3ad0141ef936f6ff6ce557bf9" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 43345 } }
I20260806 22:49:42.060303 18311 leader_election.cc:288] T 00000000000000000000000000000000 P 8a752b9c778e4c5e86ccb4389485a964 [CANDIDATE]: Term 1 election: Requested vote from peers f08a2826c5774e009b88d7ac67a4ec8e (127.18.45.253:36281), 10d437a3ad0141ef936f6ff6ce557bf9 (127.18.45.252:43345)
I20260806 22:49:42.058686 18605 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a752b9c778e4c5e86ccb4389485a964 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a752b9c778e4c5e86ccb4389485a964" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 42385 } } peers { permanent_uuid: "f08a2826c5774e009b88d7ac67a4ec8e" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 36281 } } peers { permanent_uuid: "10d437a3ad0141ef936f6ff6ce557bf9" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 43345 } } }
I20260806 22:49:42.060680 18605 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a752b9c778e4c5e86ccb4389485a964 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:49:42.060801 18417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8a752b9c778e4c5e86ccb4389485a964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f08a2826c5774e009b88d7ac67a4ec8e"
I20260806 22:49:42.060972 18417 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f08a2826c5774e009b88d7ac67a4ec8e [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:42.062264 18417 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f08a2826c5774e009b88d7ac67a4ec8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a752b9c778e4c5e86ccb4389485a964 in term 1.
I20260806 22:49:42.063844 18517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8a752b9c778e4c5e86ccb4389485a964" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "10d437a3ad0141ef936f6ff6ce557bf9"
I20260806 22:49:42.064010 18517 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 10d437a3ad0141ef936f6ff6ce557bf9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 22:49:42.065034 18517 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 10d437a3ad0141ef936f6ff6ce557bf9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a752b9c778e4c5e86ccb4389485a964 in term 1.
I20260806 22:49:42.065269 18275 leader_election.cc:302] T 00000000000000000000000000000000 P 8a752b9c778e4c5e86ccb4389485a964 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a752b9c778e4c5e86ccb4389485a964, f08a2826c5774e009b88d7ac67a4ec8e; no voters:
I20260806 22:49:42.065443 18605 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a752b9c778e4c5e86ccb4389485a964 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 22:49:42.065611 18605 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8a752b9c778e4c5e86ccb4389485a964 [term 1 LEADER]: Becoming Leader. State: Replica: 8a752b9c778e4c5e86ccb4389485a964, State: Running, Role: LEADER
I20260806 22:49:42.065718 18605 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a752b9c778e4c5e86ccb4389485a964 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a752b9c778e4c5e86ccb4389485a964" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 42385 } } peers { permanent_uuid: "f08a2826c5774e009b88d7ac67a4ec8e" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 36281 } } peers { permanent_uuid: "10d437a3ad0141ef936f6ff6ce557bf9" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 43345 } }
I20260806 22:49:42.066967 18635 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a752b9c778e4c5e86ccb4389485a964 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a752b9c778e4c5e86ccb4389485a964. Latest consensus state: current_term: 1 leader_uuid: "8a752b9c778e4c5e86ccb4389485a964" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a752b9c778e4c5e86ccb4389485a964" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 42385 } } peers { permanent_uuid: "f08a2826c5774e009b88d7ac67a4ec8e" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 36281 } } peers { permanent_uuid: "10d437a3ad0141ef936f6ff6ce557bf9" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 43345 } } }
I20260806 22:49:42.067096 18635 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a752b9c778e4c5e86ccb4389485a964 [sys.catalog]: This master's current role is: LEADER
W20260806 22:49:42.067870 18627 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8a752b9c778e4c5e86ccb4389485a964: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 22:49:42.067931 18627 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 22:49:42.075203 18638 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 22:49:42.075465 18638 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 22:49:42.077632 18351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:42.063473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40566 (local address 127.18.45.254:42385)
0806 22:49:42.063523 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:42.063525 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:42.068768 (+ 5243us) server_negotiation.cc:407] Connection header received
0806 22:49:42.068886 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:42.068887 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:42.068901 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:42.068932 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:42.069876 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:42.070681 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:42.071836 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:42.072138 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:42.072185 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:42.074103 (+ 1918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:42.074104 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:42.074105 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:42.074114 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:42.076149 (+ 2035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:42.076650 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:42.076652 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:42.076653 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:42.076719 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:42.076933 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:42.076934 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:42.076934 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:42.077081 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:42.077181 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:42.077411 (+ 230us) server_negotiation.cc:299] Negotiation successful
0806 22:49:42.077519 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 22:49:42.080328 18517 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 10d437a3ad0141ef936f6ff6ce557bf9 [term 1 FOLLOWER]: Refusing update from remote peer 8a752b9c778e4c5e86ccb4389485a964: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:49:42.080685 18635 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8a752b9c778e4c5e86ccb4389485a964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10d437a3ad0141ef936f6ff6ce557bf9" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 43345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:49:42.082306 18564 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10d437a3ad0141ef936f6ff6ce557bf9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a752b9c778e4c5e86ccb4389485a964. Latest consensus state: current_term: 1 leader_uuid: "8a752b9c778e4c5e86ccb4389485a964" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a752b9c778e4c5e86ccb4389485a964" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 42385 } } peers { permanent_uuid: "f08a2826c5774e009b88d7ac67a4ec8e" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 36281 } } peers { permanent_uuid: "10d437a3ad0141ef936f6ff6ce557bf9" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 43345 } } }
I20260806 22:49:42.082427 18564 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10d437a3ad0141ef936f6ff6ce557bf9 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:49:42.082984 18417 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f08a2826c5774e009b88d7ac67a4ec8e [term 1 FOLLOWER]: Refusing update from remote peer 8a752b9c778e4c5e86ccb4389485a964: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 22:49:42.083375 18635 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8a752b9c778e4c5e86ccb4389485a964 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f08a2826c5774e009b88d7ac67a4ec8e" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 36281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 22:49:42.084712 18560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:42.066490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50062 (local address 127.18.45.252:43345)
0806 22:49:42.066547 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:42.066550 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:42.075233 (+ 8683us) server_negotiation.cc:407] Connection header received
0806 22:49:42.075347 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:42.075348 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:42.075362 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:42.075394 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:42.076096 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:42.076849 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:42.077928 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:42.078234 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:42.078280 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:42.080233 (+ 1953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:42.080235 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:42.080236 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:42.080243 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:42.082351 (+ 2108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:42.082930 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:42.082931 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:42.082932 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:42.083016 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:42.083320 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:42.083321 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:42.083321 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:42.083474 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:42.083578 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:42.084473 (+ 895us) server_negotiation.cc:299] Negotiation successful
0806 22:49:42.084580 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 22:49:42.085052 18581 sys_catalog.cc:455] T 00000000000000000000000000000000 P f08a2826c5774e009b88d7ac67a4ec8e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a752b9c778e4c5e86ccb4389485a964. Latest consensus state: current_term: 1 leader_uuid: "8a752b9c778e4c5e86ccb4389485a964" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a752b9c778e4c5e86ccb4389485a964" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 42385 } } peers { permanent_uuid: "f08a2826c5774e009b88d7ac67a4ec8e" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 36281 } } peers { permanent_uuid: "10d437a3ad0141ef936f6ff6ce557bf9" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 43345 } } }
I20260806 22:49:42.085172 18581 sys_catalog.cc:458] T 00000000000000000000000000000000 P f08a2826c5774e009b88d7ac67a4ec8e [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:49:42.088562 18635 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a752b9c778e4c5e86ccb4389485a964 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8a752b9c778e4c5e86ccb4389485a964" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a752b9c778e4c5e86ccb4389485a964" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 42385 } } peers { permanent_uuid: "f08a2826c5774e009b88d7ac67a4ec8e" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 36281 } } peers { permanent_uuid: "10d437a3ad0141ef936f6ff6ce557bf9" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 43345 } } }
I20260806 22:49:42.088675 18635 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a752b9c778e4c5e86ccb4389485a964 [sys.catalog]: This master's current role is: LEADER
I20260806 22:49:42.089291 18564 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10d437a3ad0141ef936f6ff6ce557bf9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8a752b9c778e4c5e86ccb4389485a964" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a752b9c778e4c5e86ccb4389485a964" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 42385 } } peers { permanent_uuid: "f08a2826c5774e009b88d7ac67a4ec8e" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 36281 } } peers { permanent_uuid: "10d437a3ad0141ef936f6ff6ce557bf9" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 43345 } } }
I20260806 22:49:42.089397 18564 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10d437a3ad0141ef936f6ff6ce557bf9 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:49:42.089632 18644 mvcc.cc:205] Tried to move back new op lower bound from 7315687760197758976 to 7315687760141914112. Current Snapshot: MvccSnapshot[applied={T|T < 7315687760197758976}]
I20260806 22:49:42.089747 18645 mvcc.cc:205] Tried to move back new op lower bound from 7315687760197758976 to 7315687760141914112. Current Snapshot: MvccSnapshot[applied={T|T < 7315687760197758976}]
I20260806 22:49:42.092918 18460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:42.068507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45238 (local address 127.18.45.253:36281)
0806 22:49:42.068557 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:42.068560 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:42.078856 (+ 10296us) server_negotiation.cc:407] Connection header received
0806 22:49:42.078894 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:42.078896 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:42.078911 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:42.078970 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:42.085409 (+ 6439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:42.086128 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:42.087211 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:42.087519 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:42.087567 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:42.089521 (+ 1954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 22:49:42.089523 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 22:49:42.089523 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 22:49:42.089533 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 22:49:42.091547 (+ 2014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:42.091949 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:42.091950 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:42.091951 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:42.092011 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 22:49:42.092219 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 22:49:42.092219 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 22:49:42.092220 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 22:49:42.092362 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 22:49:42.092453 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:42.092687 (+ 234us) server_negotiation.cc:299] Negotiation successful
0806 22:49:42.092799 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 22:49:42.093784 18638 catalog_manager.cc:1347] Generated new cluster ID: 441c7bd37a1040fd8c0b24c4831efac2
I20260806 22:49:42.093845 18638 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 22:49:42.094571 18605 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a752b9c778e4c5e86ccb4389485a964 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8a752b9c778e4c5e86ccb4389485a964" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a752b9c778e4c5e86ccb4389485a964" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 42385 } } peers { permanent_uuid: "f08a2826c5774e009b88d7ac67a4ec8e" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 36281 } } peers { permanent_uuid: "10d437a3ad0141ef936f6ff6ce557bf9" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 43345 } } }
I20260806 22:49:42.094671 18605 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a752b9c778e4c5e86ccb4389485a964 [sys.catalog]: This master's current role is: LEADER
I20260806 22:49:42.095175 18581 sys_catalog.cc:455] T 00000000000000000000000000000000 P f08a2826c5774e009b88d7ac67a4ec8e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8a752b9c778e4c5e86ccb4389485a964" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a752b9c778e4c5e86ccb4389485a964" member_type: VOTER last_known_addr { host: "127.18.45.254" port: 42385 } } peers { permanent_uuid: "f08a2826c5774e009b88d7ac67a4ec8e" member_type: VOTER last_known_addr { host: "127.18.45.253" port: 36281 } } peers { permanent_uuid: "10d437a3ad0141ef936f6ff6ce557bf9" member_type: VOTER last_known_addr { host: "127.18.45.252" port: 43345 } } }
I20260806 22:49:42.095300 18581 sys_catalog.cc:458] T 00000000000000000000000000000000 P f08a2826c5774e009b88d7ac67a4ec8e [sys.catalog]: This master's current role is: FOLLOWER
I20260806 22:49:42.095489 18649 mvcc.cc:205] Tried to move back new op lower bound from 7315687760197758976 to 7315687760141914112. Current Snapshot: MvccSnapshot[applied={T|T < 7315687760197758976}]
I20260806 22:49:42.110210 18638 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 22:49:42.111116 18638 catalog_manager.cc:1504] Loading token signing keys...
I20260806 22:49:42.124642 18638 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8a752b9c778e4c5e86ccb4389485a964: Generated new TSK 0
I20260806 22:49:42.124969 18638 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 22:49:42.160027 18560 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:49:42.158188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50070 (local address 127.18.45.252:43345)
0806 22:49:42.158269 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:42.158271 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:42.159134 (+ 863us) server_negotiation.cc:407] Connection header received
0806 22:49:42.159264 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:42.159266 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:42.159296 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:42.159331 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:42.159925 (+ 594us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50070: BlockingRecv error: recv got EOF from 127.0.0.1:50070 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260806 22:49:42.161000 18460 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:49:42.158762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45250 (local address 127.18.45.253:36281)
0806 22:49:42.158816 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:42.158818 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:42.160329 (+ 1511us) server_negotiation.cc:407] Connection header received
0806 22:49:42.160444 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:42.160446 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:42.160473 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:42.160509 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:42.160911 (+ 402us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45250: BlockingRecv error: recv got EOF from 127.0.0.1:45250 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
W20260806 22:49:42.162303 18351 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:49:42.161432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40570 (local address 127.18.45.254:42385)
0806 22:49:42.161482 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:42.161485 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:42.161649 (+ 164us) server_negotiation.cc:407] Connection header received
0806 22:49:42.161755 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:42.161757 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:42.161774 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:42.161812 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:42.162203 (+ 391us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40570: BlockingRecv error: recv got EOF from 127.0.0.1:40570 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260806 22:49:42.165225 18460 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:49:42.164312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45266 (local address 127.18.45.253:36281)
0806 22:49:42.164359 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:42.164361 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:42.164643 (+ 282us) server_negotiation.cc:407] Connection header received
0806 22:49:42.164759 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:42.164760 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:42.164782 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:42.164811 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:42.165166 (+ 355us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45266: BlockingRecv error: recv got EOF from 127.0.0.1:45266 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260806 22:49:42.169050 18460 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:49:42.167938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45270 (local address 127.18.45.253:36281)
0806 22:49:42.167996 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:42.167998 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:42.168315 (+ 317us) server_negotiation.cc:407] Connection header received
0806 22:49:42.168461 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:42.168463 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:42.168479 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:42.168519 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:42.168970 (+ 451us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45270: BlockingRecv error: recv got EOF from 127.0.0.1:45270 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260806 22:49:42.173444 18460 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:49:42.172471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45282 (local address 127.18.45.253:36281)
0806 22:49:42.172514 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:42.172516 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:42.172832 (+ 316us) server_negotiation.cc:407] Connection header received
0806 22:49:42.172938 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:42.172939 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:42.172951 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:42.172987 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:42.173383 (+ 396us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45282: BlockingRecv error: recv got EOF from 127.0.0.1:45282 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260806 22:49:42.179826 18460 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:49:42.178826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45292 (local address 127.18.45.253:36281)
0806 22:49:42.178872 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:42.178874 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:42.179201 (+ 327us) server_negotiation.cc:407] Connection header received
0806 22:49:42.179308 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:42.179308 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:42.179323 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:42.179366 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:42.179765 (+ 399us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45292: BlockingRecv error: recv got EOF from 127.0.0.1:45292 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260806 22:49:42.190689 18460 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:49:42.189479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45298 (local address 127.18.45.253:36281)
0806 22:49:42.189537 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:42.189540 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:42.189875 (+ 335us) server_negotiation.cc:407] Connection header received
0806 22:49:42.189992 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:42.189994 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:42.190009 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:42.190042 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:42.190610 (+ 568us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45298: BlockingRecv error: recv got EOF from 127.0.0.1:45298 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260806 22:49:42.209559 18460 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:49:42.208453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45310 (local address 127.18.45.253:36281)
0806 22:49:42.208509 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:42.208512 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:42.208837 (+ 325us) server_negotiation.cc:407] Connection header received
0806 22:49:42.208952 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:42.208954 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:42.208969 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:42.209002 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:42.209481 (+ 479us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45310: BlockingRecv error: recv got EOF from 127.0.0.1:45310 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260806 22:49:42.244486 18460 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:49:42.243437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45312 (local address 127.18.45.253:36281)
0806 22:49:42.243508 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:42.243512 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:42.243744 (+ 232us) server_negotiation.cc:407] Connection header received
0806 22:49:42.243832 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:42.243833 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:42.243854 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:42.243915 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:42.244412 (+ 497us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45312: BlockingRecv error: recv got EOF from 127.0.0.1:45312 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
W20260806 22:49:42.311311 18460 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:49:42.310308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45324 (local address 127.18.45.253:36281)
0806 22:49:42.310401 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:42.310404 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:42.310523 (+ 119us) server_negotiation.cc:407] Connection header received
0806 22:49:42.310637 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:42.310639 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:42.310654 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:42.310735 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:42.311201 (+ 466us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45324: BlockingRecv error: recv got EOF from 127.0.0.1:45324 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260806 22:49:42.441766 18460 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:49:42.440839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45340 (local address 127.18.45.253:36281)
0806 22:49:42.440957 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:42.440960 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:42.441059 (+ 99us) server_negotiation.cc:407] Connection header received
0806 22:49:42.441138 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:42.441139 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:42.441167 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:42.441246 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:42.441683 (+ 437us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45340: BlockingRecv error: recv got EOF from 127.0.0.1:45340 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260806 22:49:42.700423 18460 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 22:49:42.699257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45342 (local address 127.18.45.253:36281)
0806 22:49:42.699333 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:42.699336 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:42.699640 (+ 304us) server_negotiation.cc:407] Connection header received
0806 22:49:42.699734 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:42.699735 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:42.699756 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:42.699823 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 22:49:42.700329 (+ 506us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45342: BlockingRecv error: recv got EOF from 127.0.0.1:45342 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 22:49:43.008853 18578 catalog_manager.cc:1259] Loaded cluster ID: 441c7bd37a1040fd8c0b24c4831efac2
I20260806 22:49:43.008970 18578 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 10d437a3ad0141ef936f6ff6ce557bf9: loading cluster ID for follower catalog manager: success
I20260806 22:49:43.009922 18578 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 10d437a3ad0141ef936f6ff6ce557bf9: acquiring CA information for follower catalog manager: success
I20260806 22:49:43.010246 18578 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 10d437a3ad0141ef936f6ff6ce557bf9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 22:49:43.045639 18600 catalog_manager.cc:1259] Loaded cluster ID: 441c7bd37a1040fd8c0b24c4831efac2
I20260806 22:49:43.045799 18600 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f08a2826c5774e009b88d7ac67a4ec8e: loading cluster ID for follower catalog manager: success
I20260806 22:49:43.046737 18600 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f08a2826c5774e009b88d7ac67a4ec8e: acquiring CA information for follower catalog manager: success
I20260806 22:49:43.047152 18600 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f08a2826c5774e009b88d7ac67a4ec8e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 22:49:43.216766 18739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:43.213746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45352 (local address 127.18.45.253:36281)
0806 22:49:43.213886 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:43.213889 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:43.213920 (+ 31us) server_negotiation.cc:407] Connection header received
0806 22:49:43.214033 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:43.214035 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:43.214120 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:43.214218 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:49:43.214398 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:43.214966 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:43.215813 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:43.216043 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:43.216122 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:43.216318 (+ 196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:49:43.216473 (+ 155us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:49:43.216573 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:43.216616 (+ 43us) server_negotiation.cc:299] Negotiation successful
0806 22:49:43.216641 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":66,"threads_started":1}
I20260806 22:49:43.221184 18745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:43.218720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50072 (local address 127.18.45.252:43345)
0806 22:49:43.218870 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:43.218872 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:43.218882 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:49:43.218931 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:43.218932 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:43.218982 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:43.219110 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:49:43.219219 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:43.219732 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:43.220420 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:43.220626 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:43.220678 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:43.220805 (+ 127us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:49:43.220924 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:49:43.220983 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:43.221018 (+ 35us) server_negotiation.cc:299] Negotiation successful
0806 22:49:43.221038 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":74,"threads_started":1}
I20260806 22:49:43.226033 18739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:43.223050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45368 (local address 127.18.45.253:36281)
0806 22:49:43.223095 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:43.223097 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:43.223267 (+ 170us) server_negotiation.cc:407] Connection header received
0806 22:49:43.223343 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:43.223344 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:43.223358 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:43.223416 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:49:43.223557 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:43.224327 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:43.225255 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:43.225459 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:43.225537 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:43.225741 (+ 204us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:49:43.225773 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:49:43.225817 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:43.225898 (+ 81us) server_negotiation.cc:299] Negotiation successful
0806 22:49:43.225921 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 22:49:43.226667 18745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:43.223048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50074 (local address 127.18.45.252:43345)
0806 22:49:43.223162 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:43.223165 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:43.223357 (+ 192us) server_negotiation.cc:407] Connection header received
0806 22:49:43.223485 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:43.223487 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:43.223501 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:43.223550 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:49:43.223737 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:43.224468 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:43.225660 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:43.225980 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:43.226063 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:43.226344 (+ 281us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:49:43.226379 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:49:43.226412 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:43.226561 (+ 149us) server_negotiation.cc:299] Negotiation successful
0806 22:49:43.226594 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260806 22:49:43.227090 18753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 22:49:43.223039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40586 (local address 127.18.45.254:42385)
0806 22:49:43.223229 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 22:49:43.223231 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 22:49:43.223241 (+ 10us) server_negotiation.cc:407] Connection header received
0806 22:49:43.223380 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 22:49:43.223381 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 22:49:43.223455 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 22:49:43.223534 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 22:49:43.223768 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:43.224552 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:43.226097 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 22:49:43.226386 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 22:49:43.226461 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 22:49:43.226667 (+ 206us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 22:49:43.226815 (+ 148us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 22:49:43.226890 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 22:49:43.226948 (+ 58us) server_negotiation.cc:299] Negotiation successful
0806 22:49:43.226976 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":65,"threads_started":1}
I20260806 22:49:43.228865 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 18243
I20260806 22:49:43.242779 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 18356
I20260806 22:49:43.253201 18615 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXSWSdD/build/release/bin/kudu with pid 18462
2026-08-06T22:49:43Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3203 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6439 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (307453 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST